Dietrich Schüller Audiovisual Archiving Visions, Challenges, Strategies Nijmegen, 29 th November 2004 PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES

History of linguistic research closely related to sound recording Audiovisual archives were initiated by research bodies  Vienna 1899  Paris and Berlin 1900  St. Petersburg 1908…  mushroomed with the advent of battery operated audio recorders in the mid-1950s  with truly portable video recorders in the 1980s video recordings became increasingly important PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Audio (and upcoming video) documents - sources proper for ethno-linguistic research  total amount audio and video carriers worldwide estimated to be million hours  amount of ethno-linguistic recordings unknown PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Visions: Future Archive usage scenarios – the topic of this workshop PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Preconditions for any visionary outlook physical existence of information linguistic data gathering did not start yesterday, but already hitherto accumulated research documents to be included considerable amounts of relevant recordings outside the world of linguistic research institutions PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Challenges carrier degradation format and system obsolescence analogue-to-digital conversion digital preservation PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Strategies spotting and ingesting language resources outside linguistic archives/institutions spotting and safeguarding unattended collections labour division: archives vs dedicated data bases PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Carrier degradation Mechanical audio carriers cylinders brittle, mouldy, extremely vulnerable shellac-78 rpm fairly stable, fragile instantaneous discs highly endangered vinyl (LP) discs fairly stable, vulnerable PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Instantaneous disc PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Magnetic tape - audio and video most endangered:  acetate tape historical audio  pigment binder break down post 1975 audio and video  MP / ME tape R-Dat, Video/Hi8, DV PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Compact Discs - Digital Versatile Discs replicated CDs + DVDsfairly stable ? recordable CDs + DVDshighly endangered DVDs: higher data density - lower data security PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Format obsolescence and availability of equipment audio format obsoleteequipment nowsoon cylinders xspecialist only coarse groove discs xspecialist only micro groove discs (vinyls) x fading out quarter inch tape xfading out micro cassettes xfading out compact cassette ? ? R-Dat xfading out MiniDisc ? ? PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Format obsolescence and availability of equipment video professional format obsoleteequipment nowsoon 2" quadruplex xspecialist only 1" xspecialist only Umatic xfading out M II xspecialist only Betacam SP ? ? D 1 xfading out D 2 xfading out D 3 xfading out D 5 x ? DigiBeta x ? IMX ? ? PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Format obsolescence and availability of equipment video home format obsoleteequipment nowsoon VCR xspecialist only Video 2000 xspecialist only Betamax xspecialist only VHS ? ? S-VHS ? ? Video8 x ? Video Hi8 x ? Digital 8 ? ? DV ? ? PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Traditional paradigm of preserving documents: preserve the original Around 1990 shift of preservation paradigm: Long term preservation of  deteriorating original documents  machines of all formats and their spare parts is hopeless! Audio (and video) preservation must be based on subsequent digital (= lossless) copying of contents PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Viable solution: Digital Mass Storage Systems (DMSSs) – robotic storage systems  self-checking  self-regenerating  self-migrating DMSS prerequisite for remote access to audiovisual collections Radio Archives introduced DMSSs since 1992 National and Research Archives following PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Storage requirement per hour preservation access audio GB<0.1GB video ~ 80GB (CCIR 601)~ 2GB (DVD) ~22GB (MPEG50, DVCPRO50) film (35mm)3.6 TB (!) photographs40-100MB/image~ 1MB/image PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Intermediary solutions- manual imitation of DMSS produce carriers free of uncorrectable errors and with a minimum of correctable errors – check each carrier and keep the error status record check error status at regular intervals, ideally of each carrier, at least of representative samples of each batch copy contents to new carrier before signal becomes irretrievable (refreshment) copy to new systems before hard- and/or software becomes obsolete (migration) PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Problems of intermediary solutions Original digital target formats R-Dat and CD-R  R-Dat close to obsolescence  CD-R (and DVD-Rs) presently unreliable Labour intensive Documents of both formats are already endangered PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Present development Avoid consumer market media (CD-Rs, DVD-Rs) and industrial (=proprietary) audio and video formats (R- Dat, DV, DigiBeta, IMX… ) Go to computer (IT) environment and openly defined file formats: Wave for audio, MXF - linear and MJPEG 2000 lossless compressed - for video Small, scalable approaches successfully under way PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Michael Risnyovszky Workstation with firewire/usb2 attached desktop RAID Small, scalable approaches towards DMSSs successfully under way To start with: A single LTO drive or a small autoloader for making backups

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Michael Risnyovszky Using 2 or more workstations File server Network switch Workstation LTO autoloader

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Michael Risnyovszky Further expanded File server with SCSI-attached drive enclosure(s) 1Gigabit server backbone Several workstations Library with 1 or 2 LTO drives

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Michael Risnyovszky 1Gigabit Server Backbone Several workstations Library with 2 LTO drives, could be connected to the SAN or directly attached to one of the file servers A small DMSS 2 or more file servers Independent, highly scalable storage unit in a SAN environment

Predominant technical task ahead: Feeding analogue and digital single carriers into a safe digital environment  time consuming  labour intensive  in demand of well maintained equipment and expert knowledge PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Transfer principles complete, unmodified transfer of originals to digital no data reduction (“compression”) - unless original was data reduced Digital resolutions for analogue originals audio  48 kHz/24 bit minimum (radio)  96/24 standard (heritage and research archives  192/24 upcoming for mechanical originals  “the worse the signal, the higher the resolution” Keep originals as long as possible PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Transfer factors cylinders, instantaneous discsopen ended 78 rpm discs1: 5 ? vinyl discsdepending on condition magnetic tape audio1:3 minimum including content description1:6 ….or: 1 hour of original tape material takes one day ! magnetic tape video1: 8–20 (!) PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Transfer may take many years, even decades Consequently  storage/handling of collections must be optimised to retard deterioration  strategies must be developed to ensure future availability of replay equipment - present greatest challenges  audio: low speed quarter inch tape machines  video “professional” player for half inch home formats PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Transfer contents is technically demanding Use modern replay equipment aligned to original (=historical) format parameters  speed  track configurations  equalisation Generally, do NOT use original recording equipment PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Quarter inch audio tape track formats 1

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller Quarter inch audio tape track formats 2 Butterfly head

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller S/N vs track width

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller Tape flux frequency response for different equalization time constants (DIN ) and recordings speeds (consumer formats)

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller Tape equalisations 1

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller Tape equalisations 2

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller Tape equalisations 3

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller Tape equalisations 4

PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller Many (most?) audio research documents need azimuth adjustment

Replay machines must be kept in good condition  spare part supply becomes an ever increasing problem  engineering knowledge and skills fading out ….transfer is more than pressing the replay button! PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Predominant financial challenge 1: cost of equipment audio:  digital mass storage systems$ 100,000++  “intermediary solutions”$ 20,000+  future “personal” DMSS$ 30,000?? video:  “video file archiving”$ PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Predominant financial challenge 2: keeping digital documents alive requires ongoing devotion and financial input professional, automated preservation: 8-12 $/GB/year manual solutions cheaper, but less secure and only viable up to the low TB range PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Predominant strategic aims 1: Spotting and ingesting language resources outside linguistic archives/institutions broadcast archives national/regional/municipial archives parliamentary archives oral history archives /collections PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Predominant strategic aims 2: Safeguarding the documents of endangered /extinct languages 80% of audiovisual materials related to ethno- linguistics are outside proper archival custody private collections and small research/cultural units, unaware of necessary preservation measures – or lacking financial and personal resources unique sources of the last 50 years are left unattended and will rot away within years spotting and preserving these collections must be organised on a great scale! PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Generally, small collections are lacking  awareness  expertise  analogue replay equipment  digital workstations  financial means for keeping digital data alive Viable solutions will require cooperative solutions Greatest problem: mistrust PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

EU funded Project TAPE objectives:  spotting small collections outside professional archival care, in cooperation with respective NGOs  awareness raising  developing training programmes and materials  organising training events Similar projects on regional and international scales needed PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Predominant strategic aims 3: Labour division between archives (in the narrower sense) content related data bases PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Archives in the narrower sense repositories of raw materials focus on high standard ingest/transfer and preservation metadata coarsely structured and restricted to otherwise unobtainable secondary information Sources for……… PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Data bases (“archives” in the wider sense) content/region etc related specifically structured time and place (school) oriented structurally between raw material and publication not responsible for high standard ingest/transfer and preservation legal and moral rights clearance viable for dedicated corpora PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Two-tier structure allows Optimising preservation and access to specialised corpora by labour division Cost of transfer from archives to be born by data bases PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Urgent recommendations Specialised data bases intensify contacts to av-archives (in the narrower sense) Coordinate activities with IASA Technical Committee IASA Cataloguing and Documentation Committee PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

Summary access to information is the aim of archiving however, access is based on the physical existence of information technical and logistic challenges to preserve information are not trivial – specifically not at the transition from analogue to digital PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller

IASA Technical Committee Standards, Recommended Practices and Strategies IASA-TC 03 The safeguarding of the Audio Heritage: Ethics, Principles and Preservation Strategy Version 2, September IASA-TC 04 Guidelines on the Production and Preservation of Digital Audio Objects, edited by Kevin Bradley Publication on IASA website forthcoming Print copies available from Ilse Assmann PHONOGRAMMARCHIV - ÖSTERREICHISCHE AKADEMIE der WISSENSCHAFTEN Dietrich Schüller

Thank you! PHONOGRAMMARCHIV - AUSTRIAN ACADEMY OF SCIENCES Dietrich Schüller