Building An Use Case Implementation With Denmark’s Broadcasting Archive Of Radio And Television (BART) Researching Fedora To Serve As Central Repository For Denmark’s State And University Library Researching Fedora, Fedora Meeting Copenhagen 09/28/05 Stephan Drescher,

picture 1 Analog Antenna Set Researching Fedora, Fedora Meeting Copenhagen 09/28/05 That’s where it all started … In an attic…

Options for System complete outsourcing of taskcomplete outsourcing of task from shelffrom shelf taylormade systemtaylormade system own developmentown development –around black box professional equipment –high-end consumer + Linux Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Nature of the data harvest process 24/7/36524/7/365 single shotsingle shot large quantities of data (220 GB a day)large quantities of data (220 GB a day) several independent input sourcesseveral independent input sources no unique identifier in or between sourcesno unique identifier in or between sources different MIME typesdifferent MIME types data needs to be evaluated and eventually corrected after 48 hoursdata needs to be evaluated and eventually corrected after 48 hours automatically ingested into repository Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Fedora figure 1 Where will be Fedora within Bart’s resource workflow? Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Hauppauge WinTV DR1 DR2 TV2 Hauppauge WinTV DR1 DR2 TV2 Hauppauge WinTV DR P1 DR P2 DR P3 PC #1 PC #1 ’ PC #2 EMC mass- storage lokal disk lokal disk lokal disk Hauppauge WinTV DR P1 DR P2 DR P3 PC #2 ’ lokal disk figure 2 Setup Capture Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Encoder setup 3 x MPEG1/2 hardware encoder = control quality and amount of data3 x MPEG1/2 hardware encoder = control quality and amount of data PCM output = lossless audio, but down- sampledPCM output = lossless audio, but down- sampled both FM Radio and TV frequencyboth FM Radio and TV frequency tuning on a single card = flexible fallback configuration parallel setup =parallel setup = backup from original signal source

picture 2 Encoder And Sampler Researching Fedora, Fedora Meeting Copenhagen 09/28/05 digital resources factory … basement

Mass storage specs TB a year (average 220 GB a day) TB a year (average 220 GB a day) Backup on tapeBackup on tape Researching Fedora, Fedora Meeting Copenhagen 09/28/05

figure 3 Mass Storage Researching Fedora, Fedora Meeting Copenhagen 09/28/05 The cooled place…

figure 3 Signal coverage Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Create recordings order Failover definitionsFailover definitions Channel definitionsChannel definitions Virtual encoder definitionVirtual encoder definition Compression definitionsCompression definitions Source definitionsSource definitions Physical encoder definitionPhysical encoder definition Valid Compression per physical encoder typeValid Compression per physical encoder type Encoder mappingsEncoder mappings –Virtual encoder –Physical encoder Abstract schedule definitionAbstract schedule definition Query definitions (where to get start/stop info)Query definitions (where to get start/stop info) Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Metadata Descriptive metadataDescriptive metadata –program information –creator, author --> Dublin Core Technical metadataTechnical metadata –encoding format –create/modified dates --> METS Logistical dataLogistical data –addresses –Path to footage e.g. --> METS Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Collect metadata Ritzau Media Service, (MSSQL database)Ritzau Media Service, (MSSQL database) DR 1, (XML stream)DR 1, (XML stream) Logfiles recording, (filesystem)Logfiles recording, (filesystem) Logfiles storage, migration (filesystem)Logfiles storage, migration (filesystem) Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Automatic harvest and ingest system Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Scheduler application ModulesModules –Bouncer –Harvester –LogSniffer –Recorder –Transformer –Inspector –Ingester –Migrator –Syncronizer Researching Fedora, Fedora Meeting Copenhagen 09/28/05

e.g. Virtual encoder definition in Recorder module Researching Fedora, Fedora Meeting Copenhagen 09/28/05

figure 4 Main Class Table BouncerAccess to system and modules Login, registrationDispatcher HarvesterCollects metadataHarvest DefinitionSyncronizer, LogSniffer Knows how to parse log files List of name-valueRecorder Schedules encoder Recording DefinitionSyncronizer TransformerArrange data Into different formats, locations e.g. From SQLResultSets to XML, METS, HTML METSWriter, Connector InspectorData quality control Checksum, Scene detection LogSniffer IngesterFeed repositoryUpload file XML stream Transformer, Fedora MigratorMove repositoryDownload file XML Stream Fedora, Transformer SyncronizerTune a taskMirror Databases Correct Data Transformer Researching Fedora, Fedora Meeting Copenhagen 09/28/05

Thank you!