Collaborative environment and workflow decomposition for remote instrumentation Roberto Pugliese ELETTRA - Sincrotrone Trieste SCpA On Behalf of the GRIDCC Collaboration BoF - Remote Instrumentation Services in Grid Environment (RISGE) OGF20 - Manchester 7-9 May 2007

BoF RISGE - OGF20 Outline The GRIDCC project  project introduction  pilot applications GRIDCC technologies  IE  VCR  Workflows Final considerations

BoF RISGE - OGF20 The GRIDCC Project The GRIDCC - Grid Enabled Remote Instrumentation with Distributed Control and Computation ( It is a 3-years EU FP6 project started in September 2004 Define and implement the “Instrument Element” to allow a standard remote access to any type of sensors and instruments Tight integration between instruments and the eInfrastructure Complex workflows integrating instruments, computational and storage resources Human Computer interaction via Virtual Control Room groupware tools that support distributed and heterogeneous teams of people in the collaborative access to the extended eInfrastructure

BoF RISGE - OGF20 The GRIDCC partners Participant nameCountry Istituto Nazionale di Fisica NucleareItaly Institute Of Accelerating Systems and ApplicationsGreece Brunel UniversityUK Consorzio Interuniversitario per TelecomunicazioniItaly Sincrotrone Trieste S.C.P.AItaly IBM (Haifa Research Lab)Israel Imperial College of Science, Technology & MedicineUK Istituto di Metodologie per l’Analisi ambientale – Consiglio Nazionale delle Ricerche Italy Universita degli Studi di UdineItaly Greek Research and Technology Network S.A.Greece

BoF RISGE - OGF20 GRIDCC Architecture Web Service Interface Execution Services WfMS WMS AS Instrument elements (IE) Instrument elements (IE) Instrument Element (IE) Compute element (CE) Compute element (CE) Computing Element (CE) Storage Element (SE) Storage Element (SE) Storage Element (SE) Problem Solver Information and Monitoring Services (IMS) Information System (BDII) Security Services Virtual Control Room Virtual Control Room Collaborative Services (CS) WMS Workload Management System WfMS WorkFlow Management System AS Agreement Service

BoF RISGE - OGF20 Execution ServicesCollaborative Environment GRIDCC Architecture Instruments GridComputational Grid IE CE SE Instrument Element Problem Solver Instrument Manager Information & Monitoring System DATA ES VCR Workflow

BoF RISGE - OGF20 GRIDCC main target areas (Large-scale) scientific experiments High energy particle physics (Radio-) Telescopes Remote process control Accelerator control (Tele-) Biomedicine Robotics Automotive Electronic microscopes Widely Sparse Instrumentation Power Grids Monitoring of the territory Monitoring of the sea Geo-hazard prediction Distributed laboratories Transportation monitoring Sensor network GRIDCC Middleware

BoF RISGE - OGF20 Remote Operations of ELETTRA: a feedback via Grid Remote Experts Operator I ESECE IM Accelerator IM Position Monitors IM Correctors See DEMO at the exibition booth 2

BoF RISGE - OGF20 Instrument Element Requirements Interactive access to allow control and monitoring Standard interface to the physical devices Fast publishing of the data acquired by instruments Fast information (logs, errors, etc.) publishing to track the behaviour of instruments and possibly solve problems Quality of Service and Advanced Reservation Computing Grid integration  Move data from instruments to the Grid storage resources (Storage Element) and viceversa  Process data produced by instruments using Grid computing resources (Computing Element) I

BoF RISGE - OGF20 The Instrument Element (IE) Instrument Control Grid Access Access to Instruments Data/Info Pubblishing Instrument Element Web Service SRM/GRIDFTP Custom (JMS,..... ) Custom + Plugins I’m here Discovery QoS Advance Reservation

BoF RISGE - OGF20 SOA accessible operations to control and monitor the instruments (via VIGS)  execute a command  get / set attributes / parameters Multi channel data production Instrument Element: multichannel I/O VIGS Commands Status Data Subscribers Grid Interaction SRM VCR Storage Element (SE) Storage Element (SE) Storage Element (SE) Logs, Errors, States, Monitors Instruments IMS Parameters ES VIGSVirtual Instrument Grid Service IMSInformation and Monitor Service SRMStorage Resource Management IE

BoF RISGE - OGF20 VIGS IMS Proxy Control Manager Data Collector Real Instruments Data Flow Control Flow State Flow Error Flow Monitor Flow The term Instrument Element describes a set of services that enable the remote control and monitoring of instruments Input Manager Event Processor FSM Engine Resource Proxy Control Manager IMS Instrument Element Architecture Resource Service IMS Problem Solver Instrument Manager Instrument Element Data Publisher Access Control Manager SRM/SE GridFTP SE IMS Instrument Protocols

BoF RISGE - OGF20 Instrument Discovery The discovery of the instruments or of the IEs is an issue when the number of elements is high We can have two use cases:  Quasi static case The number of IEs is well defined In this case a register based discovery mechanism can be used. GRIDCC tesbed is using the LCG BDII (Berkely Database Information Index ) based on LDAP The information collected in the BDII follow a GLUE schema  Dynamic case The number of IEs can change very quickly, they are very simple devices, often with poor hardware support The discovery is just use to know which are the online IEs A new approach has been developed based on Peer to Peer (P2P) protocols See DEMO at the exibition booth 2

BoF RISGE - OGF20 Instrument Element: Scalable on embedded systems FPGA PPC 405 Xilink Virtex IV Custom Board Custom Electronics 1 Gbps Ethernet Web Service GridCC IM JavaVM Linux USER INTERFACE Custom Logic Montavista JamVM Standalone Axis IE Instrument Manager

BoF RISGE - OGF20 The Virtual Control Room (VCR) The VCR provides a collaborative environment where users:  Meet and collaborate by means of groupware tools (e.g. chat, shared calendar, logbook, video conference tools)  Search, discover and browse resources such as people, instruments and other eInfrastructure resources (e.g. computing and storage)  Transparently operate with remote instruments using the GRIDCC middleware Infrastructure

BoF RISGE - OGF20 Architecture of the VCR User interface Layer providing General and Application Specific tools Services Layer providing low-level functionalities needed by the VCR Web-based portal providing some basic functionalities and support for aggregating components

BoF RISGE - OGF20 VCR in action

BoF RISGE - OGF20 Grid in the palm of your hand

BoF RISGE - OGF20 WfMS Architecture

BoF RISGE - OGF20 Remote Operations Workflows: OrbitFB

BoF RISGE - OGF20 Final considerations The GRIDCC project is integrating instruments into the “classic” computational/storage Grids. The Instrument Element allows:  virtualisation of the real instruments  support of a variety of instruments from high end to embedded devices The Virtual Control Room is a groupware tools that support distributed and heterogeneous teams of people in the collaborative access to the extended eInfrastructure The Complex workflows integrating instruments, computational and storage resources can be executed Sustainability Production installations Support new user communities