1. HEPIX Nov 1, 2000Ranger – Chuck Boeheim1 Ranger Update Chuck Boeheim SLAC

2. HEPIX Nov 1, 2000Ranger – Chuck Boeheim2 History Nicemon Patrol 2.0 Ranger 3.0 – Never released Ranger 4.0

3. HEPIX Nov 1, 2000Ranger – Chuck Boeheim3 New Approach Previous versions interpreted a rules file –This added complexity and errors New version is a Perl toolkit –No scanner, parser, or interpreter needed –Rich library of tests and actions provided –Nearly anything else can be expressed in Perl

4. HEPIX Nov 1, 2000Ranger – Chuck Boeheim4 Observations An observation is an object with the values describing one entity –E.g. one process, one filesystem, one file Observations may be saved between runs –Deltas calculated from previous runs –History may be accumulated Observations may be made about aggregate entities, too –E.g. cumulative processes for one user, numbers of daemons running.

5. HEPIX Nov 1, 2000Ranger – Chuck Boeheim5 Collectors Collectors return observations to the calling script Also responsible for saving them between calls New collectors easily created by overriding the base Collector class

6. HEPIX Nov 1, 2000Ranger – Chuck Boeheim6 Built-in Collectors PSCollector – processes DFCollector – file systems FileCollector – file sizes, md5 signatures NSCollector – listening sockets SystemCollector – load averages, uptime, number of users, processes Watch – tail a file or process (in development)

7. HEPIX Nov 1, 2000Ranger – Chuck Boeheim7 Basic Structure use PSCollector; apply pscollector => ruleset {rule { cpu > 50 and time > 60*MIN } action { log; mail LIMIT, user; kill }; rule { cpu > 50 and time > 20*MIN } action { log; mail HOG, user; nice }; }; Includes code for collector Generates one observation per call Calls collector repeatedly, feeds observation to ruleset Brackets a set of rules The first rule that succeeds ends the ruleset Rules return true or false Actions are blocks done when true

8. HEPIX Nov 1, 2000Ranger – Chuck Boeheim8 Rules A rule statement is followed by a block that must return true or false rule { (pct > 50 and delta > 10) or (pct > 75 and delta > 5) } sub chk { pct > $_[0] and delta > $_[1] } rule { chk(50,10) or chk(75,5) } Each rule has an associated action

9. HEPIX Nov 1, 2000Ranger – Chuck Boeheim9 Actions An action statement is followed by a block. Any perl statements allowed within the block. rule { pct > 90 } action { system 'cleanup' };

10. HEPIX Nov 1, 2000Ranger – Chuck Boeheim10 Built-in Actions Built-in actions operate on the current observation: –nice – kill – restart –mail – page – log –mcons action { kill; mail RUNAWAY, user; page BADPROC, 'admin', log };

11. HEPIX Nov 1, 2000Ranger – Chuck Boeheim11 Messages The Message statement stores text in a library and tags it: Message HOG => <<'EOF'; You have been using $pct% of one CPU on $host. EOF Actions like mail, page, and log retrieve message by their tag: mail HOG, user; Messages are evaluated each time they are retrieved.

12. HEPIX Nov 1, 2000Ranger – Chuck Boeheim12 Observation Values Functions provide values from current observation –name – size – user –pct – time – delta –loadavg – numcpu – numusers –Numprocs Can be used in either rules or actions. Prototyped unary functions need no args

13. HEPIX Nov 1, 2000Ranger – Chuck Boeheim13 Scheduling Tasks every 10*MIN => background { apply pscollector => ruleset { … };

14. HEPIX Nov 1, 2000Ranger – Chuck Boeheim14 Units Constants are provided to provide natural expression of units: –Time: SEC, MIN, HOUR, DAY –Size: KB, MB, GB Examples cpu > 2*HOUR size > 1*MB and delta > 100*KB

15. HEPIX Nov 1, 2000Ranger – Chuck Boeheim15 Repetition Suppression Many actions take a parameter to suppress multiple triggers for the same observation mail HOG, user, 6*HR;

16. HEPIX Nov 1, 2000Ranger – Chuck Boeheim16 MD5 Watching The file collector can observe MD5 checksums of files –Either supply a list of known good checksums: rule { name eq '/lib/security/pam_afs.so' and md5change('bf1501489fc0bb9bf0052f624558aed4', '13f50f924bbe64758c6400c4fd412ae7') } –Or let it record the first one as the base: rule { name eq '/usr/lib/libc.a' and md5change }

17. HEPIX Nov 1, 2000Ranger – Chuck Boeheim17 Filtering a File Intended to replace swatch for many purposes apply watch('/var/log/messages') => ruleset { rule { /error/ } action { mcons; log }; rule { /ecache parity error/ } action { page …}; };

18. HEPIX Nov 1, 2000Ranger – Chuck Boeheim18 Example: Watch for Bad Guys apply pscollector => ruleset { # Processes that we want to know about. rule { name =~ /crack|irc|eggd|satan/ } action { mail PROC_REPORT,'boeheim',1*DAY; mcons ALERT; log 1*DAY }; } Message PROC_REPORT => '$name is running on $host';

19. HEPIX Nov 1, 2000Ranger – Chuck Boeheim19 Example: Watch Daemons apply pscollector([sshd ools]) ruleset { # Only execute this ruleset for summary records by name. return unless summary eq 'BYNAME'; rule { number == 0 } action { my $mailto = admin; perform ruleset { rule { name eq "sshd" } action { restart '/usr/etc/sshd' }; rule { name eq 'ools' } action { $mailto = 'oomonitor' }; }; mail NO_DAEMON, $mailto; mcons ALERT; log 6*HR; }

20. HEPIX Nov 1, 2000Ranger – Chuck Boeheim20 Example: Watch Filesystems apply dfcollector() => ruleset { rule { name =~ m(^/(var|tmp)) } action {rule { size 99 } action { mail FSFULL, admin, 6*HR; log 2*HR }; rule { size 95, [90,1], [80,5], [50+50] } action { mcons 2*HR; log 2*HR }; };