1. Andrew Hanushevsky8-October-99 1 Objectivity Open File System http://www.slac.stanford.edu/~abh/HEPiX99-oofs/ HEPiX October 8, 1999 Andrew Hanushevsky SLAC Computing Services Produced under contract DE-AC03-76SF00515 between Stanford University and the Department of Energy

2. Andrew Hanushevsky8-October-99 2 The oofs View SLAC Designed & Developed Veritas IBM DOE Objectivity Filesystem Logical Layer Filesystem Physical Layer Filesystem Implementation Database Protocol Layer User

3. Andrew Hanushevsky8-October-99 3 Different Implementations

4. Andrew Hanushevsky8-October-99 4 Common Disk Cache Management ooss_Stage ooss_MigrPurg ooss_rrs ooss_PreStage rx_hpss, etc

5. Andrew Hanushevsky8-October-99 5 Directory Layout /var/adm/ ooss/ logs/ ams:amslog_day.n prestage:slog migrate:mlog purge:plog migrpurg, STOPMIGR, STOPPSTG, STOPPURG pftp/keyfile PreStageQ/stageRequests.0, stageRequests.1 /usr/etc/ooss/executables, utilities, & configuration files refreshed from /afs/slac.stanford.edu/package/hpss/objectivity/distrib refreshed from /afs/slac.stanford.edu/package/hpss/objectivity/distrib /tmpooss_migrpurg.pid, ooss_prestage.pid /objy/databases production/production objectivity database

6. Andrew Hanushevsky8-October-99 6 Distribution Directory I n /afs/slac.stanford.edu/package/hpss/objectivity/distrib u RestartAMS- Stop and start the ams u StartAMS- Start the ams (many options + fd limit setting) u StopAMS- Stop the ams u TraceAMS- Dynamically set/reset trace flags u amshpss- Executable u fs_stat- df-like command used by ooss_Stage u keeplog- Log handling used by StartAMS u ooss_MigrPurg- Migration/Purge manager u ooss_PreStage- Prestage manager u ooss_Stage- Staging manager u ooss_ffail- Find all “.fail” files

7. Andrew Hanushevsky8-October-99 7 Distribution Directory II n /afs/slac.stanford.edu/package/hpss/objectivity/distrib u ooss_mklk- Create a “.lock” file u ooss_nolk- Find files that have no “.lock” file u perlx64- Perl with large file support ( warning uses slac libs ) u pftp_client- Parallel ftp client u pud- Perl Universal Daemon u pudc- Perl Universal Daemon Client u rrs_Client.pm- Resource Reservation System client library u rxhpss- HPSS client gateway

8. Andrew Hanushevsky8-October-99 8 Distribution Directory Documentation n /afs/slac.stanford.edu/package/hpss/objectivity/distrib/doc u Read_Me F Contains notes on the latest changes in the distribution directory F Cumulative history u Future resting place for man pages u Current documentation is “the code” F Needs to be changed F Config file documentation can be found on the web in a presentation http://www.slac.stanford.edu/~abh/oofs_desc/

9. Andrew Hanushevsky8-October-99 9 New Features n ooss_MigrPurg u Support for multiple filesystems F Solves the 1TB filesystem limit F Still needs support in staging manager and ams (coming) u Elimination of e-mail message flooding F Messages are now periodically batched u Small bug fixes n ooss_PreStage u Elimination of e-mail message flooding F Messages are now periodically batched n ooss_ffail u New command to find “.fail” files n ooss_mklk u Small bug fixes in group code check

10. Andrew Hanushevsky8-October-99 10 New Features II n amshpss u Support for running multiple amshpss processes F Controlled via StartAMS -s service_name The service name establishes the port the ams will use The service name must be present in /etc/services F Client can then communicate with appropriate ams Using /etc/services on client side to direct the client via ooams-2 entry –Orthogonal solution which requires same number of ams’ everywhere Using special connect() wrapper to selectively direct client to other ams’ –Can run any kind of configuration –Need to relink client programs

11. Andrew Hanushevsky8-October-99 11 Support for Multiple Filesystems n Solves the 1TB limit u Handled via symbolic links n Completely upward compatible u Standard processing used unless turned on via config option F ooss.cachefs path [, lopurge, hipurge ] n Requires changes in many components u ooss_MigrPurg u ooss_Stage u amshpss n Requires new commands to manually handle cache u ooss_audit u ooss_cache {cp | mv | rm} u ooss_rebuild

12. Andrew Hanushevsky8-October-99 12 Example of Multiple Filesystem Support /cache1/databases\mydbfile /databases/mydbfile /cache2 /cache3 symlink Index Area Optional data cache Default data area Data Area Any number Any Size Chosen based on free space in LRU order

13. Andrew Hanushevsky8-October-99 13 All in the pursuit of performance

14. Andrew Hanushevsky8-October-99 14 Conclusion n New features driven by labs and BaBar u Requires periodic resynchronization n Probably should have a “users” group u Collection of requirements u Description of changes u Information sharing n Probably should meet periodically u HEPiX or some other forum? u Frequency? u Should we do this at all?