Monitoring data db Monitoring data database Paola Grosso SCS Networking The idea (from Warren) is to store the data from various monitoring programs into.

Slides:

Advertisements

Similar presentations

Copyright © 2003 Pearson Education, Inc. Slide 8-1 The Web Wizards Guide to PHP by David Lash.

Advertisements

Copyright © SoftTree Technologies, Inc. DB Tuning Expert.

Module 13: Performance Tuning. Overview Performance tuning methodologies Instance level Database level Application level Overview of tools and techniques.

Company Confidential 1 © 2005 Nokia DBUpgradeTool_ ppt / / JMa A Database Upgrade Tool Nokia Networks Jukka Maaranen.

Connect. Communicate. Collaborate WI5 – tools implementation Stephan Kraft October 2007, Sevilla.

1 SLAC Internet Measurement Data Les Cottrell, Jerrod Williams, Connie Logg, Paola Grosso SLAC, for the ISMA Workshop, SDSC June,

GGF-BerlinSLAC Web Services SLAC Web Service Paola Grosso (SLAC) Presented by Eric Boyd (Internet2)

CSC 2720 Building Web Applications Database and SQL.

8/6/ ITE 370: SQL Stored Procedures. 8/6/ Stored Procedures A stored procedure is A stored procedure is a collection of SQL statements saved.

HORIZONT 1 TWS/WebAdmin Tips & Tricks HORIZONT Software for Datacenters Garmischer Str. 8 D München Tel ++49(0)89 /

Network Management: SNMP

DAT702.  Standard Query Language  Ability to access and manipulate databases ◦ Retrieve data ◦ Insert, delete, update records ◦ Create and set permissions.

CERN IT Department CH-1211 Genève 23 Switzerland t Streams new features in 11g Zbigniew Baranowski.

Database Design and Introduction to SQL

IST Databases and DBMSs Todd S. Bacastow January 2005.

Batak Programming Language

CLEO’s User Centric Data Access System Christopher D. Jones Cornell University.

A Guide to SQL, Eighth Edition Chapter Three Creating Tables.

Benefits of PL/SQL. 2 home back first prev next last What Will I Learn? In this lesson, you will learn to: –List and explain the benefits of PL/SQL –List.

DATABASE. A database is collection of information that is organized so that it can easily be accessed, managed and updated. It is also the collection.

Stanford Linear Accelerator Center November 15, 2000Lee Ann Yasukawa1 Archive Data to ORACLE The Prototype PEPII model.

1 Oracle Database 11g – Flashback Data Archive. 2 Data History and Retention Data retention and change control requirements are growing Regulatory oversight.

Introduction on R-GMA Shi Jingyan Computing Center IHEP.

On-Demand View Materialization and Indexing for Network Forensic Analysis Roxana Geambasu 1, Tanya Bragin 1 Jaeyeon Jung 2, Magdalena Balazinska 1 1 University.

Computer Science 101 Database Concepts. Database Collection of related data Models real world “universe” Reflects changes Specific purposes and audience.

Stored Procedures, Triggers, Program Access Dr Lisa Ball 2008.

Stanford Linear Accelerator Center R. Hall/L. Yasukawa1 EPICS Collaboration Mtg May 21, 2002 Oracle Storage for the Channel Archiver Managing Channel Archiver.

Windows NTFS Introduction to Operating Systems: Module 15.

Chapter 2: SQL – The Basics Objectives: 1.The SQL execution environment 2.SELECT statement 3.SQL Developer & SQL*Plus.

Stanford Linear Accelerator Center R. D. Hall1 EPICS Collaboration Mtg Oct , 2007 Oracle Archiver Past Experience Lessons Learned for Future EPICS.

Module 5: Upgrading to SQL Server 7.0. Overview Planning an Upgrade Preparing to Upgrade Verifying the Upgrade Setting a Compatibility Level.

CS1100: Data, Databases, Queries Action Queries CS11001Advanced Queries.

Copyright © 2010 Certification Partners, LLC -- All Rights Reserved Perl Specialist.

October 27, 2015 Atlas Monitoring Infrastructure in Grid Environment Richard Baker Dantong Yu Brookhaven National Lab.

DataGrid Wide Area Network Monitoring Infrastructure (DWMI) Connie Logg February 13-17, 2005.

Tips and Tricks for Managing and Administering your Enterprise Project Management Server Solution Mike Joe / Karthik Chermakani Software Test Engineer.

Database Agility Pramod Sadalage ThoughtWorks. © ThoughtWorks, Inc.© Agile.Database:// Introduction The Traditional Database Continuous changes.

8 1 Chapter 8 Advanced SQL Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.

Web Server Administration Chapter 7 Installing and Testing a Programming Environment.

Local Monitoring at SARA Ron Trompert SARA. Ganglia Monitors nodes for Load Memory usage Network activity Disk usage Monitors running jobs.

Course FAQ’s I do not have any knowledge on SQL concepts or Database Testing. Will this course helps me to get through all the concepts? What kind of.

14 Copyright © Oracle Corporation, All rights reserved. SQL Workshop.

Copyright © 2003 ProsoftTraining. All rights reserved. Perl Fundamentals.

Download the website to work locally Tool: Surf Offline 1.0 Create PERL program to extract website structure information and storage.

Module 4 Introduction ADO.NET.

1 An Implementation of the Profile Document. Presented at the June 24-27, 2003.

Distributed Logging Facility Castor External Operation Workshop, CERN, November 14th 2006 Dennis Waldron CERN / IT.

ECMM6018 Enterprise Networking For Electronic Commerce Tutorial 6 CGI/Perl and databases.

FTS monitoring work WLCG service reliability workshop November 2007 Alexander Uzhinskiy Andrey Nechaevskiy.

Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Author etc Dashboard Latency monitoring (Update) Alexandre Beche.

SQL. What is a database? G a collection of data G Usually consists of entities and relations G An entity is an individual “object” that exists and is.

Database Management COP4540, SCS, FIU Database Trigger.

CS1100: Data, Databases, Queries Action Queries CS11001Advanced Queries.

Level 1-2 Trigger Data Base development Current status and overview Myron Campbell, Alexei Varganov, Stephen Miller University of Michigan August 17, 2000.

CS 440 Database Management Systems Stored procedures & OR mapping 1.

Tool Support for Testing Classify different types of test tools according to their purpose Explain the benefits of using test tools.

Rationale Databases are an integral part of an organization. Aspiring Database Developers should be able to efficiently design and implement databases.

Retele de senzori EEMon Electrical Energy Monitoring System.

Chapter 27 Network Management Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Databases and DBMSs Todd S. Bacastow January

Module 11: File Structure

GO! with Microsoft Office 2016

GO! with Microsoft Access 2016

Building A Web-based University Archive

Multitier Architecture, MySQL & PHP

PHP / MySQL Introduction

Data Management Innovations 2017 High level overview of DB

SLAC monitoring Web Services

Presentation transcript:

Monitoring data db Monitoring data database Paola Grosso SCS Networking The idea (from Warren) is to store the data from various monitoring programs into an Oracle database for: –Faster retrieval –Easier data selection –Trend studies –Archiving purposes

Monitoring data db Oracle Database We are using the new Oracle 9 db, called SLACPROD, instead of the older one, SLAC_TCP. This will allow us to grow the data size more easily and possibly take advantage of new Oracle feature as partitioning (quoting Ian MacGregor)

Monitoring data db Measurements The data we are storing comes from 3 different sources: RIPE_TT measurement ABwE measurements IEPM-BW measurements There are separate tables for each one of the data sources.

Monitoring data db RIPE_TT table The data is collected in real time from the data that has been received from the various RIPE TT boxes: SQL> descr RIPE_TT: Name Null? Type ALIASNAME VARCHAR2(20) NODENAME VARCHAR2(30) VERSION NUMBER(38) IPDEST VARCHAR2(30) PORTDEST NUMBER(38) IPSDR VARCHAR2(30) PORTSDR NUMBER(38) ARRIVALTIME VARCHAR2(20) DELAY NUMBER(63) NTPSRC VARCHAR2(20) NTPDEST VARCHAR2(20) SEQUENCE NUMBER(38) NTPERRSDR NUMBER(63) NTPERRDEST NUMBER(63)

Monitoring data db ABWE table The data stored in this table is extracted in real time from the ABwE measurements performed every minute: SQL> descr abwe; Name Null? Type ALIASNAME VARCHAR2(30) NODENAME VARCHAR2(30) IPADDRESS VARCHAR2(30) TIMESTAMP NUMBER(38) ABW NUMBER(63) XTR NUMBER(63) DBCAP NUMBER(63) AVABW NUMBER(63) AVXTR NUMBER(63) AVDBCAP NUMBER(63)

Monitoring data db IEPM-BW_ROUTE table IEPM_BW_ROUTE contains the data for the ctrace,rtrace and trace measurements SQL> descr IEPM_BW_ROUTE; Name Null? Type ALIASNAME VARCHAR2(30) NODENAME VARCHAR2(30) IPADDRESS VARCHAR2(30) TIMESTAMP NUMBER(38) TOOLNAME VARCHAR2(30) ROUTENUMBER NUMBER(38) ROUTE VARCHAR2(1024)

Monitoring data db IEPM_BW table This table contains the data from the iperf,qiperf,gftp and bbftp measurements. SQL> descr IEPM_BW; Name Null? Type ALIASNAME VARCHAR2(30) NODENAME VARCHAR2(30) IPADDRESS VARCHAR2(30) TIMESTAMP NUMBER(38) TOOLNAME VARCHAR2(30) THROUGHPUT NUMBER(63) STREAM NUMBER(38) WINDOW_SIZE NUMBER(38)

Monitoring data db Current status We are debugging the tables layout and the scripts that store the data. We have already stored: RIPE_TT ~350k entries ABWE ~50k entries IEPM_BW ~7k entries IEPM_BW_ROUTE ~20k entries

Monitoring data db Code structure The scripts that perform the data archiving are located in the AFS area: /afs/slac/g/www/www-iepm/tools/netrat/tools There 3 Perl scripts and one Perl module for the actual archiving, and 3 Shell scripts for the management of the programs.