Presenting and aggregating network statistics with Stager

Slides:



Advertisements
Similar presentations
ICmyNet.Flow Network Traffic Analysis System If You Want to See Your Net
Advertisements

17th February, 2000 by Maciej Korzeniowski (CERN-IT-IA-MI) 1 Oracle Discoverer Product Presentation  This is an ad hoc query and analysis tool for.
ESafe Reporter V3.0 eSafe Learning and Certification Program February 2007.
Good afternoon. My name is Marek Pawłowski
Overview of Database Administrator (DBA) Tools
Connect. Communicate. Collaborate Click to edit Master title style MODULE 1: perfSONAR TECHNICAL OVERVIEW.
Monitoring a Large-Scale Network: Selecting the Right Tool Sayadur Rahman United International University & Network Manager, Financial Service.
The Visibility Information Exchange Web System is a database system and set of online tools originally designed to support the Regional Haze Rule enacted.
16 months…. The Visibility Information Exchange Web System is a database system and set of online tools originally designed to support the Regional Haze.
Monitoring backbone networks Manuel ubredu, Valeriu Vraciu – RoEduNet Chiinău, September 9, 2014.
Router Architectures An overview of router architectures.
HWg-STE: simple WEB thermometer
VMware vCenter Server Module 4.
Hyrax Installation and Customization Dan Holloway James Gallagher.
These materials are licensed under the Creative Commons Attribution-Noncommercial 3.0 Unported license (
Module 18 Monitoring SQL Server 2008 R2. Module Overview Monitoring Activity Capturing and Managing Performance Data Analyzing Collected Performance Data.
1 Network Statistic and Monitoring System Wayne State University Division of Computing and Information Technology Information Technology.

Channel Archiver Stats & Problems Kay Kasemir, Greg Lawson, Jeff Patton Presented by Xiaosong Geng (ORNL/SNS) March 2008.
Net Optics Confidential and Proprietary Net Optics appTap Intelligent Access and Monitoring Architecture Solutions.
Descriptive Data Analysis of File Transfer Data Sudarshan Srinivasan Victor Hazlewood Gregory D. Peterson.
Flow monitoring (Netflow) Status report 12/10/2006 Alessadro Inzerilli - GARR.
INDIANAUNIVERSITYINDIANAUNIVERSITY TransPAC2 Security John Hicks TransPAC2 Indiana University 22nd APAN Conference – Singapore 20-July-2006.
Session 2 Security Monitoring Identify Device Status Traffic Analysis Routing Protocol Status Configuration & Log Classification.
Hsu Chun-Hung Network Benchmarking Lab
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.
Network Management Tool Amy Auburger. 2 Product Overview Made by Ipswitch Affordable alternative to expensive & complicated Network Management Systems.
NetFlow: Digging Flows Out of the Traffic Evandro de Souza ESnet ESnet Site Coordinating Committee Meeting Columbus/OH – July/2004.
workshop eugene, oregon What is network management? System & Service monitoring  Reachability, availability Resource measurement/monitoring.
What is Sure Stats? Sure Stats is an add-on for SAP that provides Organizations with detailed Statistical Information about how their SAP system is being.
Graphing and statistics with Cacti AfNOG 11, Kigali/Rwanda.
Database tools An SQL layer around CDF. Overview Current SPENVIS database Limitations SQL wrapped CDF Advantages Demonstration.
JDeveloper 10g and Oracle ADF Business Components Getting the Most Out of Your Data Avrom Roy-Faderman Senior Programmer November, 2005.
Open-Eye Georgios Androulidakis National Technical University of Athens.
Chapter 6: Using The Windows Performance and Reliability Monitor.
Application Development
OPeNDAP Hyrax Harnessing the power of the BES OPeNDAP Hyrax Back-End Server Patrick West
CERN General Infrastructure Services Department CERN GS Department CH-1211 Geneva 23 Switzerland Db Futures Workshop
1 Copyright © 2005, Oracle. All rights reserved. Following a Tuning Methodology.
Distributed Logging Facility Castor External Operation Workshop, CERN, November 14th 2006 Dennis Waldron CERN / IT.
ECHO A System Monitoring and Management Tool Yitao Duan and Dawey Huang.
The ECOST Web-based platform for data providers and for data users.
Cofax Scalability Document Version Scaling Cofax in General The scalability of Cofax is directly related to the system software, hardware and network.
Use cases for BnF broad crawls Annick Lorthios. 2 Step by step, the first in-house broad crawl The 2010 broad crawl has been performed in-house at the.
Splitting a Database: How and Why Roger Smeaton. CONTENTS Splitting a database - what is it? Why do it? How is it done?
Network Management Presentation HP Openview. OpenView Network Node Manager (NNM) Overview How it works Capabilities Technical and business benefits Summary.
NetFlow Analyzer Best Practices, Tips, Tricks. Agenda Professional vs Enterprise Edition System Requirements Storage Settings Performance Tuning Configure.
1 Experiences Deploying and Operating a Large-Scale Monitoring Infrastructure Network Monitoring Workshop October 21, 2009 Arne Øslebø
1 PerfSONAR implementation using Netconf Network Monitoring Workshop October 21, 2009 Arne Øslebø
Monitoring Storage Systems for Oracle Enterprise Manager 12c
System Monitoring with Lemon
© 2002, Cisco Systems, Inc. All rights reserved.
Network Management week 11
Presented By: #NercompPDO3
Network Administration CNET-443
Monitoring Storage Systems for Oracle Enterprise Manager 12c
Network Monitoring System
HWg-STE: simple WEB thermometer
Umbraco /Product Finder
RM3G: Next Generation Recovery Manager
Chapter 15 Introduction to Rails.
Intelligent Integration Ambulatory Blood Pressure
PlanetFlow The PlanetLab Network Auditing Service Mark Huang
Topic 11 Lesson 1 - Analyzing Data in Access
Working Knowledge Training
Features Overview.
Hao Hu, Luo Qi, Fazhi Qi IHEP 22 Mar. 2018
BIOPAMA Data Management
© 2002, Cisco Systems, Inc. All rights reserved.
Presentation transcript:

Presenting and aggregating network statistics with Stager Network Monitoring Workshop October 21, 2009 Arne Øslebø arne.oslebo@uninett.no

Stager A web based tool for presenting and aggregating most types of network statistics Store high level reports in database Stable version NetFlow Qflow Development version Qstream Mping Asmping Rude/Crude SNMP http://software.uninett.no/stager/

Template based All data is stored in a table XML based template specifies how frontend should present the data bps, pps, %, temperature, IP address etc. Transformations Table, overview and matrix. Multiple views for each transformation. Specifies which data types should be visible New statistics can be added without code changes to the frontend Built in functions divideandmultiply, persec, bitpersec, concat etc.

Stager architecture Netconf Backend1 Backend1 SQL SQL Frontend www www Backend2 Backend2 Netconf Netconf Backend3 Backend3 Raw netflow

Backend robustness Graceful handling of database downtime Store reports to files Insert into database when possible Avoid multiple instances of the same backend Wait for previous instances that are still processing the raw data Detect dead locks and memory starvations Supports anycast for NetFlow Raw NetFlow data automatically processed by collector

Netflow in Stager Reports generated every hour Aggregated to day, week, month and year Available reports Summary Interface Src/Dst AS/IP/Port Protocol Mulicast Src AS Src IP Groups (Dst IP) Possible to view raw NetFlow data for more details Reports follow links, not interfaces

Our NetFlow setup 34 routers 439 interfaces ~25Gb of raw Netflow data every day Anycast >600 millions entries in a single table >1TB database size

Overview report

Detailed report

Graph