PAWN Progress July 06, 2006

Overview of changes New flexible environment for setting up and managing interactions between producers and the archive Domains to organize accounts, record organization, and packages Definable roles that can be flexibly combined and assigned to accounts Interfaces for designing package builders and archival resource gateways

Components Producer Managed Archive Managed Management Server Producer data suppliers Receiving Server Distributed Archive Schedule Request Authentication Package Information Ingestion Status Validation Services

Overall Organization Producers organized into domains, each domain containing a record schedule negotiated with the archive. Each domain contains a hierarchy of the types of data and record sets (convenient groupings from the record schedule). An end-user operates within a domain with record sets associated with the account.

Package Workflow 1. Client selects a record set to use as a package template. 2. A package is built locally and then transferred to a PAWN receiving server. 3. Optionally lock package to signal complete submission. 4. Review and possible reject items. 5. Transfer items from PAWN into final archive. 6. Remove package from PAWN.

Record Organization Previous version had one hierarchy with attachment points for items as leaf nodes. Did not allow for linking of related leaf nodes Hierarchy performed multiple roles, record organization and administrative organization. Current version based on Record Sets. Separate administrative structure and record structure. Record Sets are template packages.

Record Organization Each domain contains a record schedule Record schedule is a hierarchy containing authorities as endpoints Domains also contain an organizational hierarchy. Offices, projects, etc. Record Sets group of authorities from the record schedule attached to a point in the record hierarchy. Have access permissions Presented to producers as package templates

College of Sciences Domain  Office of the Dean  Chemistry  Mathematics  Physics  Computer Science o Business Office o Research Groups o Labs … Record Sets Record Schedule Administrative oStrategic and Performance Plans oAppointment and Promotion oPolicies and Committees oAlumni Affairs Financial oContracts and Grants oPayroll oDonations Publication Reports oTechnical Reports - Archiving Rules oPresentations oPosters Record Set  Name: Research Results  Note: Reports, presentations, and other published research results  Allowed Accounts Record Schedule Mapping  Presentations o Presentations  Technical Reports o Technical Reports Domains  Offices of the President and Vice-Presidents  College of Sciences  College of Engineering  College of Medicine  College of Arts and Humanities  College of Behavioral and Social Sciences  ….. Record Set Sample

Flexible Account Roles Previous version had fixed accounts, producer, manager and administrator. Current version allows actions in PAWN to be grouped into roles. Each account is assigned a role. Sample actions in PAWN Record Set/Schedule management Package creation/deletion/modification Account management

SAML Usage SAML Assertions are issued by managers Contain manager namespace, domain, username Contain list of allowed actions by the client Contain client’s public key (holder-of-key) Signed by manager SAML Assertions authenticate and authorize a client for archive-side services. Package Management Calls Archive Management Calls Administrative Metadata Calls ArchiveProducer Call Overlap

Sample SAML Assertion umiacs:toaster urn:oasis:names:tc:SAML:1.0:cm:holder-of-key MIIDxjCCAy+gAwIBAgIDEAACMA0GCSqGSIb3DQEB.... view create modify...

SAML Assertion (cont) r7C4oNmlf4h8cXi1dGU+MIGmGbM= Rstfd1HKTe68WLQrgAvmS5hDm7SVbXnEgMlotW3aiu.... MIIDyjCCAzOgAwIBAgIDEAABMA0GCSqGSIb3DQ....

Package Creation Packages are built using a Record Set as a template. Each category in a Record Set has a hierarchy of manifests attached. Manifests are an abstraction of underlying METS documents Custom package builders use manifest interface. Manifest  Namespace  Type  Descriptive Name Data  Type  Descriptive Name  Bits Metadata … Manifest … Metadata  Type  Bits  Name

Package Builders Default Builder Create files and folders Attach descriptive metadata to files or folders ICDL Builder Create ‘books’ with dublin core metadata Uses ICDL database as source for book list and metadata

Package Scheduling and Submission Scheduler decides which receiving server to store a package Condor classad system used Receiving server periodically publishes available resources Client request space. Client Receiver Scheduler 2. Evaluate classad 1. Space Requirements 3. Create Reservation 4. Allocated Server 5. Package Transfer Receiver Classads

Publishing into Archival Resources PAWN provides an interface for registering gateways into archival resources Gateways provide: Configuration gui Client gui Mover to transfer data from PAWN to archive PAWN provides: Configuration storage Access to all items in a package Access to contextual information about a package Infrastructure for storing and loading gateway drivers.

SRB Publishing PAWN Package SRB Gateway SRB 5. GUID or Path3. Package Items Archival Context 4. Package Items PAWN Client 2. SRB Path & item list PAWN Scheduler 1. SRB Configuration

Screenshots Client Interface Configuration Interface Resulting Log Entry

XFDU publishing Create XFDU compatible Information Packet. XFDU is similar to METS. Separate data definitions from structural information Similar file attributes (size, checksum, etc..) PAWN mapping InformationPackageMap contains ContentUnits to recreate the hierarchy of data in a PAWN package. DataObjects register individual files. XFDU manifest and data files combined to form an Information Package.

Demo