Presentation is loading. Please wait.

Presentation is loading. Please wait.

First EELA Conference, Santiago(Cile), 4th September,2006 - GridCC: Real-time Instrumentations Grids.

Published byPatricia Poole Modified over 8 years ago

Similar presentations

Presentation on theme: "First EELA Conference, Santiago(Cile), 4th September,2006 - GridCC: Real-time Instrumentations Grids."— Presentation transcript:

1 First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) GridCC: Real-time Instrumentations Grids A real-time interactive GRID to integrate instruments, computational and information resources widely spread on a fast WAN Andrea Petrucci Istituto Nazionale di Fisica Nucleare Laboratori Nazionali di Legnaro, Legnaro Italy

2 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 2 Overview The GridCC Project: Introduction Bringing Instrument into the Grid: the Instrument Element The GridCC Test-bed: Pilot application Instrument Instrumentation Fast Instrument Communication Channel Standard Grid Interaction

3 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 3 General on the GridCC Project Participant nameCountry Istituto Nazionale di Fisica NucleareItaly Institute Of Accelerating Systems and Applications Greece Brunel UniversityUK Consorzio Interuniversitario per Telecomunicazioni Italy Sincrotrone Trieste S.C.P.AItaly IBM (Haifa Research Lab)Israel Imperial College of Science, Technology & Medicine UK Istituto di Metodologie per l’Analisi ambientale – Consiglio Nazionale delle Ricerche Italy Universita degli Studi di UdineItaly Greek Research and Technology Network S.A. Greece It is a 3 years project. Started the 1st September 2004 Funded by EU in the Frame Program 6 10 Partners from 3 EU Countries + (Israel) About 40 people engagged www.gridcc.org

4 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 4 The Grid Technologies extend the limit of a single computer (center) Grid Technologies User Interface Grid Gateway ComputingElement StorageElement ComputingElement Current Grid technologies offer unlimited computational power and storage capacity for scientific research and business activities However concrete use cases demand more close interaction between various types of instruments accessible from the Grid elements

5 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 5 Extending the Grid Concepts Instruments Grid Computational Grid GridCC Project + Data for Model Calculations Predictions The goal is to build a widely distributed system that is able to remotely controls and monitors complex instruments

6 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 6 Storage Elements Storage Elements Computing Element Computing Element Instrument Element Instrument Element: global scenario Computing Element Storage Element Instrument Element Instrument Element Existing Grid Infrastructures Web Service Interface Virtual Control Room Virtual Control Room Exec. Service WfMS WMS AgrS User direct Action Indirect Action

7 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 7 Virtual Control Room (VCR) All end user access is via the VCR Instrument elements (IE) The IE is a virtualization of the real physical instrument Instrument elements (IE) Instrument elements (IE) Of course there may be many IEs Compute and Storage Elements (with advanced reservation) Storage Element (SE) Compute element (CE) Of course Many CEs and SEs Storage Element (SE) Compute element (CE) Storage Element (SE) Compute element (CE) Collaborative Services (CS) Virtual Control Room (VCR) Users generally not working alone Direct access to IE SE (and CE) possible but often not desirable Information and Monitoring Services (IMS) “Fast” all pervasive messaging system Information System (IS) Slowly updating information Security Services Security is essential to the success of the project Global Problem Solver Watching (via the IMS) for problems anywhere in the system and acting to resolve them. Execution Services More complex workflows, including advanced reservation and QoS guarantees, allowed The GridCC Architecture

8 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 8 IE Requirements Web Services Instrument Element Any Protocol or physical connection Sensor Network Instrument Grid Computing Element ElementStorage Computing InstrumentElement W E F A B C D 1: Provide a uniform access to the physical device 2: Allow a standard grid access to the instruments 3: Allow the cooperation between different instruments that belong to different VOs

9 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 9 Instrument Element: a Black Box IE VIGS Instrument Instrumentation Fast communication channel  The term Instrument Element describes a set of services that provide the needed interface and implementation that enables the remote control and monitoring of physical instruments. Grid Interaction Data Mover Instruments Quick Answers to the previous slide: 1)The VIGS provide the a uniform instrument instrumentation way 2)The fast communication channel disseminate the acquired information between instruments 3)The Data Mover provide a standard Grid Interface in order to be accessed by others Grids components like the SE and the CE

10 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 10 Instrument Element Architecture Virtual Instrument Grid Service (VIGS) Resource Service Inf & Mon Service Problem Solver Instrument Manager Instrument Element Data Mover IMS Proxy Control Manager Data Collector Real Instruments Data Flow Control Flow State Flow Error Flow Monitor Flow The IE entry point is constitued by a Web Service interface called Virtual Instrument Grid Service(VIGS) The VIGS allows users to access and control the specific physical Instrument from any part in the world using a VCR. Access Control Manager execute() getState() create() destroy() Input Manager Event Processor FSM Engine Resource Proxy Control Manager

11 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 11 Instrument Element Implementations Resource Service Inf & Mon Service Problem Solver Instrument Manager Instrument Element Data Mover Access Control Manager The IE components are typically implemented into a fully equipped Machines (e.g. dual core cpus, large memory, large disks, etc). This is true for RS, IMS and PS. For IM (and DM) there are 2 possibilities, according to the application type: IM implemented in a fully equipped machine IM embedded into the instrument that should be controlled IM RS IMS IM Embedded Web Service

12 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 12 Instrument Manager IMS Proxy Data Collector Instrument Manager Input Manager Event Processor FSM Engine Resource Proxy Control Manager Instruments Data Flow State Flow Error Flow Monitor Flow Control Flow Customizable Plug-in modules to interface to the instruments IM is composed by 3 main components:  Control Manager:  Input Manager : It handles all the input events of the IM. These includes commands from GUIs or other IMs,errors/state/log/monitor messages.  Event Processor: It handles all the incoming message and decide where to send them. It has processing capability  FSM: A finite state machine is implemented.  Resource Proxy: It handles all the outgoing connections with the resources.  Data Collector: It get data from the controlled instruments and make them available to the data mover. A local storage of the data is even foreseen.  IMS Proxy: It receives error/state/log/monitor information from the controlled resources and forward them to IMS.

13 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 13 Resource Service Architecture The Resource Service (RS) handles all the resources of an IE and manages their partition (if any). A resource can be any hardware or software component involved in the IE (instruments, Instrument Managers, IMS components) RS stores the configuration data of the resources and download them to resource target when necessary Resources can be discovered, allocated and queried. It is the responsibility of the RS to check resource availability and contention with other active partitions when a resource is allocated for use. A periodic scan of the registered resources keeps the configuration database up to date. RS is interfaced to the WMS Discovery Manager Subscribe Manager Partition&Lock Manager Configuration Manager Available Resources Partition Definitions Configuration Definitions RS Data Bases Partition/Configuration retrieve methods Partition and Lock setting methods Configuration setting methods Discovery methods

14 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 14 Information and Monitor System (IMS) PUBLISHERS (Instruments nodes) SUBSCRIBERS Errors Log info Monitor State The Information and Monitor Service (IMS) collects messages and monitor data coming from GRID resources and supporting services and stores them in a database. There are several types of messages collected from the sub-systems. The messages are catalogued according to their type, severity level and timestamp. Data can be provided in numeric formats, histograms, tables and other forms. The IMS collects and organizes the incoming information in a database and publishes it to subscribers. These subscribers can register for specific messages categorized by a number of selection criteria, such as timestamp, information source and severity level. Instrument Manager Instruments Instrument Manager Instruments Instrument Manager Instruments

15 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 15 Problem Solver IMS Proxy Control Manager Instrument Manager Pub/Sub IMS Proxy Control Manager Instrument Manager IMS Proxy Control Manager Instrument Manager IMS Proxy Control Manager Instrument Manager DB Data Mining Tools Algorithms evaluations : Rule Induction, Tree, Functions, Lazy, Clusters and Associative State Flow Error Flow Monitor Flow On Line Analisys Problem Solver Step 1 The control manager can perform an autonomous recovery action where the cost for the determination it is not so heavy. Step 2 Persistent information can be analyzed in order to extract knowledge Step 3 On-line information can be analyzed in order to detect possible malfunctions

16 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 16 Standard Grid Interaction: Data Mover The task of this element is to get data from the “data collector” of the IM Data can be accessed via:  Web service interface for generic data dump (e.g. slow storage, spy stream, etc.)  grid storage element (SE) and available CEs can access to the data via an SRM Interface  Http server and TCP communication for high performance had-hoc data transfer The Data Mover exposes its methods to the IE web service and can be instrumented itself as an instrument. Instrument Resources Data Mover Data Collector IM IE Web Service Interface: get_data() SRM interface Http Server and TCP/IP raw socket Data Collector IM Data Collector IM

17 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 17 Main GRIDCC Pilot Applications CMS detector - CERN Power Grid - Brunel Intrusion Detection System – NTUA/IASA Far remote control of accelerator - Elettra

18 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 18 Main GridCC Pilot Applications: The Run Control prototype is controlling and monitoring a Data Acquisition (DAQ) system of the High Energey Physical Experiment CMS at the CERN Large Handron Collider.

19 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 19 The CMS Data Acquisition O(10 4 ) distributed Objects to  control  configure  monitor On-line diagnostics and problem solving capability Highly interactive system (human reaction time - fraction of second) World Wide distributed monitor and control 2 10 7 electronics channels 40 MHz 100 Hz

20 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 20 CMS Prototype: IEs at work Det 1 DAQ TTS FedBuilderRuBuilder FilterFarm Trigger TOP GTPe DAQ Detector 1 8 - GridCC middleware used for CMS MTCC (Magnet Test and Cosmic Challenge) - 11 Instrument Elements with a hierarchical topology - Instruments are in these case Linux hosts where the cms on-line software is running - More than 100 controlled hosts CMS Instrument Elements DAQ IE Instrument Managers

21 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 21 Power Grid Instrument Manager Instrument Element... Virtual Control Room Virtual Control Room Gas Solar Power Grid V.O The Power Grid prototype is controlling and monitoring power generators (Solar, gas..)

22 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 22 Remote Operation of an Accelerator Elettra Synchrotron Far remote operation of an accelerator facility in­volves the planning of accelerator operations, the maintenance of the accelerator and its trouble­shooting, the repair of delicate equipment, understanding and pushing performance limitations, performing studies, performing commissioning and set ups and rou­tine operations.

23 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 23 The other GridCC pilot applications Meteorology (Ensemble Limited Area Forecasting) Device Farm for the Support of Cooperative Distributed Measurements in Telecommunications and Networking Laboratories Geo-hazards: Remote Operation of Geophysical Monitoring Network (see first slides) Medical Devices need a close loop between the data acquisition and the output result

24 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 24 Conclusion The GridCC project is integrating instrument into traditional computational/storage Grids. IEs need an high interaction and interactivity between itself and the users. The GridCC IE implementation is currently installed in heterogeneous applications

25 Andrea Petruccci, First EELA Conference, Santiago(Cile), 4th September,2006 (andrea.petrucci@lnl.infn.it - andrea.petrucci@cern.ch) - 25 Question..... Thank you for your time Acknowledgement: The GridCC project is supported under EU FP6 contract 511382. More information: www.gridcc.org

Download ppt "First EELA Conference, Santiago(Cile), 4th September,2006 - GridCC: Real-time Instrumentations Grids."

Similar presentations

Gaetano Maron, Information Event, Brussel 26-27 May 2005 1 GRIDCC A realtime interactive GRID to integrate instruments, computational and information resources.

Gaetano Maron, Information Event, Brussel May GRIDCC A realtime interactive GRID to integrate instruments, computational and information resources.
Distributed Data Processing

Distributed Data Processing
DORII: Training for Pilot Application Evaluation Paolo Gamba Matteo Lanati.

DORII: Training for Pilot Application Evaluation Paolo Gamba Matteo Lanati.
A Java Architecture for the Internet of Things Noel Poore, Architect Pete St. Pierre, Product Manager Java Platform Group, Internet of Things September.

A Java Architecture for the Internet of Things Noel Poore, Architect Pete St. Pierre, Product Manager Java Platform Group, Internet of Things September.
Performance analysis of a Grid-based Instrumentation Device Farm Luca Berruti 1, Franco Davoli 1, Stefano Vignola 1, Sandro Zappatore 1 1 CNIT – University.

Performance analysis of a Grid-based Instrumentation Device Farm Luca Berruti 1, Franco Davoli 1, Stefano Vignola 1, Sandro Zappatore 1 1 CNIT – University.
CHEP 2012 – New York City 1.  LHC Delivers bunch crossing at 40MHz  LHCb reduces the rate with a two level trigger system: ◦ First Level (L0) – Hardware.

CHEP 2012 – New York City 1.  LHC Delivers bunch crossing at 40MHz  LHCb reduces the rate with a two level trigger system: ◦ First Level (L0) – Hardware.
F. Lelli, Summer School on Grid Computing, Ischia, July 20, 2006 GridCC: Real-time Instrumentations Grids A real-time interactive GRID to integrate instruments,

F. Lelli, Summer School on Grid Computing, Ischia, July 20, 2006 GridCC: Real-time Instrumentations Grids A real-time interactive GRID to integrate instruments,
DataGrid is a project funded by the European Union 22 September 2003 – n° 1 EDG WP4 Fabric Management: Fabric Monitoring and Fault Tolerance

DataGrid is a project funded by the European Union 22 September 2003 – n° 1 EDG WP4 Fabric Management: Fabric Monitoring and Fault Tolerance
23/04/2008VLVnT08, Toulon, FR, April 2008, M. Stavrianakou, NESTOR-NOA 1 First thoughts for KM3Net on-shore data storage and distribution Facilities VLV.

23/04/2008VLVnT08, Toulon, FR, April 2008, M. Stavrianakou, NESTOR-NOA 1 First thoughts for KM3Net on-shore data storage and distribution Facilities VLV.
Universität Stuttgart Universitätsbibliothek Information Retrieval on the Grid? Results and suggestions from Project GRACE Werner Stephan Stuttgart University.

Universität Stuttgart Universitätsbibliothek Information Retrieval on the Grid? Results and suggestions from Project GRACE Werner Stephan Stuttgart University.
Gaetano Maron, ANW, Halifax, November 2004 1 GRIDCC A realtime interactive GRID to integrate instruments, computational and information resources widely.

Gaetano Maron, ANW, Halifax, November GRIDCC A realtime interactive GRID to integrate instruments, computational and information resources widely.
Multi-media Computers and Computer Networks. Questions ? Media is used for ………………… Multimedia computer is capable of integrating ………………………………….. OCR stands.

Multi-media Computers and Computer Networks. Questions ? Media is used for ………………… Multimedia computer is capable of integrating ………………………………….. OCR stands.
Test Of Distributed Data Quality Monitoring Of CMS Tracker Dataset H->ZZ->2e2mu with PileUp - 10,000 events ( ~ 50,000 hits for events) The monitoring.

Test Of Distributed Data Quality Monitoring Of CMS Tracker Dataset H->ZZ->2e2mu with PileUp - 10,000 events ( ~ 50,000 hits for events) The monitoring.
EGEE-II INFSO-RI-031688 Enabling Grids for E-sciencE www.eu-egee.org EGEE II - Network Service Level Agreement (SLA) Establishment EGEE’07 Mary Grammatikou.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE II - Network Service Level Agreement (SLA) Establishment EGEE’07 Mary Grammatikou.
A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster

A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster
CMS Michele Gulmini, Cern, DAQ Weekly 07/05/2002 1 RCMS – Plan of work Michele Gulmini DAQ Weekly 7th May 2002.

CMS Michele Gulmini, Cern, DAQ Weekly 07/05/ RCMS – Plan of work Michele Gulmini DAQ Weekly 7th May 2002.
Active Monitoring in GRID environments using Mobile Agent technology Orazio Tomarchio Andrea Calvagna Dipartimento di Ingegneria Informatica e delle Telecomunicazioni.

Active Monitoring in GRID environments using Mobile Agent technology Orazio Tomarchio Andrea Calvagna Dipartimento di Ingegneria Informatica e delle Telecomunicazioni.
F. Lelli, Advanced Training Workshop on Instruments and Sensors on the Grid,Trieste, April 25, 2007 Instrument Elements at Work Imperial Brunel Elettra.

F. Lelli, Advanced Training Workshop on Instruments and Sensors on the Grid,Trieste, April 25, 2007 Instrument Elements at Work Imperial Brunel Elettra.
The Run Control and Monitoring System of the CMS Experiment Presented by Andrea Petrucci INFN, Laboratori Nazionali di Legnaro, Italy On behalf of the.

The Run Control and Monitoring System of the CMS Experiment Presented by Andrea Petrucci INFN, Laboratori Nazionali di Legnaro, Italy On behalf of the.
JCOP Workshop September 8th 1999 H.J.Burckhart 1 ATLAS DCS Organization of Detector and Controls Architecture Connection to DAQ Front-end System Practical.

JCOP Workshop September 8th 1999 H.J.Burckhart 1 ATLAS DCS Organization of Detector and Controls Architecture Connection to DAQ Front-end System Practical.

Similar presentations

Ads by Google