The GrangeNet NOC Greg Wickham

Contents Goals Design Features Architecture Implementation Future Conclusion

Contents Goals Design Features Architecture Implementation Future Conclusion

Goals Central repository where all information about the network is available Monitoring of the network including availability and utilisation Access control to ensure that the right people have access to the right data Leverage existing utilities where practical

Contents Goals Design Features Architecture Implementation Future Conclusion

Design Features Easy to use –Intuitive Web Framework aids rapid development Modular Construction –Extensible Security from the ground up Unified configuration –Mimimise replication of data

Contents Goals Design Features Architecture Implementation Future Conclusion

Architecture Log file monitoring Security server Database backend Historical configurations

Architecture

db Database contains information on: devices; connections; interfaces; as numbers; prefixes; subnets; members

Architecture devices db logfiles Devices connected to the network syslog to the NOC

Architecture devices logwatcher db logfiles Logwatcher process monitors log files; messages from devices defined in the db are stored in the db

Architecture devices logwatcher pancho db configs logfiles Logwatcher notices a configuration change; pancho is used to download the config and store in a RCS

Architecture devices logwatcher pancho utilisation db configs logfiles datacollector Datacollector periodically examines SNMP counters on device; stores in RRDTool db

Architecture devices logwatcher pancho utilisation db configs logfiles tacasip datacollector Tacacs authentication is used for all Cisco devices; no shared enable secret

Architecture devices logwatcher pancho conf report utilisation db configs logfiles tacasip datacollector Configuration change reports are mailed to the noc-manager nightly

Architecture devices logwatcher pancho conf report utilisation www db configs logfiles tacasip datacollector Web server is primary interactive portal

Architecture devices logwatcher pancho conf report utilisation weathermap www db configs logfiles tacasip datacollector Weathermap renders graphic of total network utilisation

Architecture devices logwatcher pancho conf report utilisation weathermap www db configs logfiles tacasip datacollector grapher Grapher renders historical graphs

Architecture devices logwatcher pancho conf report utilisation weathermap www Config generator db configs logfiles tacasip datacollector grapher ‘cookie cutter’ configurations are generated

Architecture devices logwatcher pancho conf report utilisation weathermap www Config generator db configs logfiles tacasip datacollector grapher

Contents Goals Design Features Architecture Implementation Future Conclusion

Leverage Open Source Software

The Perl Libraries Unified configuration format RCS Object Orientated HTML Dynamic Web Framework

Unified configuration format Similar to Apache COORD-RELATIVE edge1.act COORD-SYSTEM polar #polar coords are ( distance, angle ) COORD 130,-45 LABEL TEXT apac RADIUS 20

RCS Based on CPAN RCS Added support for ‘-p’ option –Diff into array

Object Orientated HTML Much easier to render dynamic HTML Automatic closure Inline back patching my $html = new HTML::OO; $html->table()->tr()->td(‘Hello World’); my $a = $html->a( { href => ‘link’ } ); $a->print(“Click here”);

Dynamic Web Framework All web pages are rendered in real time Concept of Modules and Hierarchy Integrated cookie based authentication –Integrated with LDAP Simplifies writing new modules, especially when the new module: –Is required to integrate with existing modules –Needs authentication protection –Can leverage existing library routines

The Database Uses PostgreSQL Contains information describing the network –Who is connected? –Where are they connected? –What devices are connected? –What interfaces are on the devices? –Asset management

The Data Collector Extracts list of devices to monitor from database MIBS automatically monitored by defined device classes –Automatically tracks hardware For interfaces – interface name is used For others – SNMP index is used (ie: cpu etc)

The tacacs server Tacacs used as authentication protocol Backended into AARNet LDAP Stateful protocol - No shared enable password Accounting (console command logging) Supports some NAS features

The Logwatcher Monitors logged output from network devices –Logged messages are screen for sanity (bad formatting etc.) –Inserted into the database –Any configuration change triggers a download of the configuration –Store configuration files stripped of ‘secrets’ Stored in RCS

The Web Server Uses Apache – Perl embedded interpreter (mod_perl) – mod_ssl / OpenSSL (Security) Mod_perl –Increases Apache memory footprint Flexibility (access internal API) Performance (db access) –Decreases CGI script startup time

The Configuration Reporter Every night all registered devices are checked –Are running-config and startup-config the same? –Report any changes made in the last 24 hours –Summary report ed to noc-manager

The weathermapper Goal – At a glance network status Configuration file defines the weathermap –Nodes: Defines nodes and connections –Map: Defines nodes to draw Automatically generates: –Full map (all data rendered as requested) –Quick mapping Background pre-rendered Foreground drawn over background –Meta data for ISMAP clicking

The weathermapper Goal – At a glance network status Configuration file defines the weathermap –Nodes: Defines nodes and connections –Map: Defines nodes to draw Automatically generates: –Full map (all data rendered as requested) –Quick mapping Background pre-rendered Foreground drawn over background –Meta data for ISMAP clicking

The weathermapper Embedded Nagios status

The weathermapper Throughput on link Percentage utilisation indicated by colour

The weathermapper Legend for colurs / percentage utilisation

The weathermapper Links down

The weathermapper Connection status highlight (In progress)

The Support Cast Suite of scripts that: –Collect SNMP data –Check interface names against db –Analyse Nagios status (scraping)

The Result Live demo GrangeNet Survey

Contents Goals Design Features Architecture Implementation Future Conclusion

The Future Nagios? – Where does it fit –Automatic config generation (from db) –Many similar queries to the same device aren’t efficient More reporting –Link statistics –Bandwidth utilisation Shim layer –Scraping currently used –Improved efficiency if APIs are well defined Ie: Nagios

Contents Goals Design Features Architecture Applications Implementation Future Conclusion

Perl – Rapid application development Proven to be very useful Centralised data –Describes the entire network –The one source of information

Questions?