Andrew Hanushevsky8-October-99 1 Objectivity Open File System HEPiX October 8, 1999 Andrew Hanushevsky SLAC Computing Services Produced under contract DE-AC03-76SF00515 between Stanford University and the Department of Energy

Andrew Hanushevsky8-October-99 2 The oofs View SLAC Designed & Developed Veritas IBM DOE Objectivity Filesystem Logical Layer Filesystem Physical Layer Filesystem Implementation Database Protocol Layer User

Andrew Hanushevsky8-October-99 3 Different Implementations

Andrew Hanushevsky8-October-99 4 Common Disk Cache Management ooss_Stage ooss_MigrPurg ooss_rrs ooss_PreStage rx_hpss, etc

Andrew Hanushevsky8-October-99 5 Directory Layout /var/adm/ ooss/ logs/ ams:amslog_day.n prestage:slog migrate:mlog purge:plog migrpurg, STOPMIGR, STOPPSTG, STOPPURG pftp/keyfile PreStageQ/stageRequests.0, stageRequests.1 /usr/etc/ooss/executables, utilities, & configuration files refreshed from /afs/slac.stanford.edu/package/hpss/objectivity/distrib refreshed from /afs/slac.stanford.edu/package/hpss/objectivity/distrib /tmpooss_migrpurg.pid, ooss_prestage.pid /objy/databases production/production objectivity database

Andrew Hanushevsky8-October-99 6 Distribution Directory I n /afs/slac.stanford.edu/package/hpss/objectivity/distrib u RestartAMS- Stop and start the ams u StartAMS- Start the ams (many options + fd limit setting) u StopAMS- Stop the ams u TraceAMS- Dynamically set/reset trace flags u amshpss- Executable u fs_stat- df-like command used by ooss_Stage u keeplog- Log handling used by StartAMS u ooss_MigrPurg- Migration/Purge manager u ooss_PreStage- Prestage manager u ooss_Stage- Staging manager u ooss_ffail- Find all “.fail” files

Andrew Hanushevsky8-October-99 7 Distribution Directory II n /afs/slac.stanford.edu/package/hpss/objectivity/distrib u ooss_mklk- Create a “.lock” file u ooss_nolk- Find files that have no “.lock” file u perlx64- Perl with large file support ( warning uses slac libs ) u pftp_client- Parallel ftp client u pud- Perl Universal Daemon u pudc- Perl Universal Daemon Client u rrs_Client.pm- Resource Reservation System client library u rxhpss- HPSS client gateway

Andrew Hanushevsky8-October-99 8 Distribution Directory Documentation n /afs/slac.stanford.edu/package/hpss/objectivity/distrib/doc u Read_Me F Contains notes on the latest changes in the distribution directory F Cumulative history u Future resting place for man pages u Current documentation is “the code” F Needs to be changed F Config file documentation can be found on the web in a presentation

Andrew Hanushevsky8-October-99 9 New Features n ooss_MigrPurg u Support for multiple filesystems F Solves the 1TB filesystem limit F Still needs support in staging manager and ams (coming) u Elimination of message flooding F Messages are now periodically batched u Small bug fixes n ooss_PreStage u Elimination of message flooding F Messages are now periodically batched n ooss_ffail u New command to find “.fail” files n ooss_mklk u Small bug fixes in group code check

Andrew Hanushevsky8-October New Features II n amshpss u Support for running multiple amshpss processes F Controlled via StartAMS -s service_name The service name establishes the port the ams will use The service name must be present in /etc/services F Client can then communicate with appropriate ams Using /etc/services on client side to direct the client via ooams-2 entry –Orthogonal solution which requires same number of ams’ everywhere Using special connect() wrapper to selectively direct client to other ams’ –Can run any kind of configuration –Need to relink client programs

Andrew Hanushevsky8-October Support for Multiple Filesystems n Solves the 1TB limit u Handled via symbolic links n Completely upward compatible u Standard processing used unless turned on via config option F ooss.cachefs path [, lopurge, hipurge ] n Requires changes in many components u ooss_MigrPurg u ooss_Stage u amshpss n Requires new commands to manually handle cache u ooss_audit u ooss_cache {cp | mv | rm} u ooss_rebuild

Andrew Hanushevsky8-October Example of Multiple Filesystem Support /cache1/databases\mydbfile /databases/mydbfile /cache2 /cache3 symlink Index Area Optional data cache Default data area Data Area Any number Any Size Chosen based on free space in LRU order

Andrew Hanushevsky8-October All in the pursuit of performance

Andrew Hanushevsky8-October Conclusion n New features driven by labs and BaBar u Requires periodic resynchronization n Probably should have a “users” group u Collection of requirements u Description of changes u Information sharing n Probably should meet periodically u HEPiX or some other forum? u Frequency? u Should we do this at all?