1. Bulk Data Retrieval Implemented as MIB extensions Bryan Levin snmp@AllegroNetworks.com draft-ietf-eos-snmp-bulkdata-01.txt MPLS, 18-mar-2002 David Battle dbattle@cisco.com

2. Problem Statement SNMP/UDP datagrams are not efficient for Bulk DataSNMP/UDP datagrams are not efficient for Bulk Data –UDP has limited payload size SNMP/TCP never gained mass acceptanceSNMP/TCP never gained mass acceptance –Double ACK’ing is inefficient SNMP lacks native ability for atomic snapshotsSNMP lacks native ability for atomic snapshots –get-bulk is great but still not atomic No SNMP way to sync shapshot to many NMS’sNo SNMP way to sync shapshot to many NMS’s –Each NMS’s polled data would be different (time skew)

3. Goals Define semantics for building synthetic tables –Based on real table “slices” Define semantics for atomic table snapshots –Snapshot happens entirely on local agent Define semantics for bulk file transfers –Using existing encryption and compression Define data file format to be as simple as possible –ASCII flatfiles are very easy to write and read Allow remote control of locally queued snapshots

4. Proposed Solution Based on Existing Ideas Many vendors use “SNMP-triggered” file transfers –Understood & accepted method of sending large datasets. Additional features added –Atomic snapshot semantic –Data slicing creates synthetic tables –Compression/Encryption –File redundancy –Both Push and Pull models supported –Local snapshot queue management Not addressed –Row-based filtering/suppression

5. Benefits Leverages existing File Transfer Protocols –FTP, SCP, HTTP are well debugged and deployed. –Minimal engineering needs be done Lower risk since less new code has to be developed. Quick interoperability with minimal fuss –Many fileservers already have ftpd and sshd configured. Reduced NMS polling & interactions –The NMS for initial Bulk Definition (via an SNMP SET). –File transfer reliable & on time (agent self-triggering).

6. Synthetic Tables Similar to D. Perkin’s getcols & Nalin Pai’s idea –Allow the user to define a table in terms of columns from existing (agent) tables –Once a synthetic table has been defined, it can be reused without the need to redefine it in every incoming PDU –This works for tables that are logical ‘augmentations’ of each other (classic example: ifTable and ifXtable)

7. Setup Procedure Overview 1) Read agent’s snapshot capabilities (scalars) 2) Define a synthetic table (SliceTable entries) –List of column OIDs from real tables 3) Define fileserver table entries –Where and how to get the snapshot to the server 4) Request a snapshot to a local file –Refer to an entry in (2) and an entry in (3) 5) Monitor results table –Check status, abort, retry, delete-row

8. MIB Scalars NMS should check these before initiating SETs Describes the agent’s snapshot mib capabilities –acFileEncoding (xml, ascii) –acFileCompression (none, bzip, gzip) –acXferProtocol (cp, ftp, scp)

9. MIB Table #1 Slice Table Describe a subset of columns from real tables –sliceIndex, sliceSubId (major, minor instance fields) –sliceColumnOID (column to include in synthetic table definition) –sliceSnmpContext (community string to select mgmt context) –sliceColumnDisplayHint (printf-style %s, %d strings) –sliceAdminString (for NMS use) –sliceEntryStatus (RowStatus)

10. MIB Table #2 Xfer Table Describe Where and How it gets there –xferIndex, xferSubId (instance fields) –xferHostIpType, xferHostIpAddr (network address of fileserver) –xferProtocol (cp, ftp, scp) –xferWriteControl (failIfExists, ovewrite) –xferFilePath (path on fileserver to save files) –xferAuthUsername, xferAuthPassword (remote authentication) –xferAdminString (for NMS use) –xferEntryStatus (RowStatus)

11. MIB Table #3 Snapshot Table Request a snapshot to local non-volatile storage –index, sliceIndex, xferId (instance fields) –snapshotSnapFileName (name used to save snapshot, local+remote) –snapshotFileEncoding (ascii, xml) –snapshotFileCompression (gzip, bzip, none) –snapshotStartTime, snapshotCompletionTime (timestamps) –snapshotFileSize (of actual snapshot file) –snapshotState (status of local snapshot event) –snapshotAdminString (for NMS use) –snapshotEntryStatus (RowStatus)

12. MIB Table #4 XferControl Table Monitor & Manage queued transfer requests –xferCtlIndex, sliceIndex, xferIndex, xferSubId (instance fields) –xferCtlStartTime (timestamp at begin of file transfer) –xferCtlCompletionTime (timestamp at ending of file transfer) –xferCtlPercentXferred (amount of file that has transferred so far) –xferCtlStatus (misc status + abort, retry, delete)

13. sliceTable xferTable snapshotTable xferControlTable

14. # 992796710, index, ifHCInOctets, ifInErrors, ifHCOutOctets, ifOutErrors 1, 9279383, 25, 8443465, 2 5, 2794335, 0, 1646610, 0 10, 733837, 1, 731334, 28 14, 2793883, 22, 2789734, 397 23, 794335, 6, 564610, 19 192.168.1.1:/home/abc/ 192.168.1.2:/home/abc/ localhost:/pub/stats/ Iftable_030102.gz Result of Snapshot cp ftp scp

15. Server-side Bulk Table Format CSV’s are natural for application import File contents are viewable by standard text editors # timestamp, column, column,… inst,val,val,val,… … Empty cells are indicated by “null” values (,,) Quotes are allowed, escaped by reverse slash “\”

16. Bulk Table File Examples ifTable # 992796710, ifDescr, ifOperStatus, ifLastChange, ifHCInOctets, ifHCOutOctets 1, fxp0, up(1), 992793883, 2443465, 24435775 5, ge-6/1/0.0, up(1), 992794335, 16496610, 14634989 10, fddi-0, down(2), 992733837, 2342334, 2863772 14, fxp0.0, up(1), 992793883, 38734, 39773 23, so-7/0/0, up(1), 992794335, 16496610, 14634989 ipCidrRouteTable # 992796710, ipCidrRouteIfIndex, ipCidrRouteType, ipCidrRouteProto 0.0.0.0.0.0.0.0.0.192.168.4.254 14, remote(4), netmgt(3) 192.168.1.0.255.255.255.0.0.192.168.4.254, 5, remote(4), netmgt(3) 192.168.9.195.255.255.255.255.0.0.0.0.0, 23 reject(2), ospf(13)

17. Summary Low impact to existing management infrastructure File compression keeps xfer overhead low Datasets as ASCII comma-delimited fields Rendundant fileservers (multiple file copies) allowed Local atomic snapshot semantic Data slices from similarly-instanced real tables Timing (snapshot, upload) determined by agent Leverages other MIBS (eg, DISMAN)