1. Collaborative environment and workflow decomposition for remote instrumentation Roberto Pugliese ELETTRA - Sincrotrone Trieste SCpA On Behalf of the GRIDCC Collaboration BoF - Remote Instrumentation Services in Grid Environment (RISGE) OGF20 - Manchester 7-9 May 2007

2. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Outline The GRIDCC project  project introduction  pilot applications GRIDCC technologies  IE  VCR  Workflows Final considerations

3. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it The GRIDCC Project The GRIDCC - Grid Enabled Remote Instrumentation with Distributed Control and Computation (www.gridcc.org)www.gridcc.org It is a 3-years EU FP6 project started in September 2004 Define and implement the “Instrument Element” to allow a standard remote access to any type of sensors and instruments Tight integration between instruments and the eInfrastructure Complex workflows integrating instruments, computational and storage resources Human Computer interaction via Virtual Control Room groupware tools that support distributed and heterogeneous teams of people in the collaborative access to the extended eInfrastructure

4. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it The GRIDCC partners Participant nameCountry Istituto Nazionale di Fisica NucleareItaly Institute Of Accelerating Systems and ApplicationsGreece Brunel UniversityUK Consorzio Interuniversitario per TelecomunicazioniItaly Sincrotrone Trieste S.C.P.AItaly IBM (Haifa Research Lab)Israel Imperial College of Science, Technology & MedicineUK Istituto di Metodologie per l’Analisi ambientale – Consiglio Nazionale delle Ricerche Italy Universita degli Studi di UdineItaly Greek Research and Technology Network S.A.Greece

5. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it GRIDCC Architecture Web Service Interface Execution Services WfMS WMS AS Instrument elements (IE) Instrument elements (IE) Instrument Element (IE) Compute element (CE) Compute element (CE) Computing Element (CE) Storage Element (SE) Storage Element (SE) Storage Element (SE) Problem Solver Information and Monitoring Services (IMS) Information System (BDII) Security Services Virtual Control Room Virtual Control Room Collaborative Services (CS) WMS Workload Management System WfMS WorkFlow Management System AS Agreement Service

6. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Execution ServicesCollaborative Environment GRIDCC Architecture Instruments GridComputational Grid IE CE SE Instrument Element Problem Solver Instrument Manager Information & Monitoring System DATA ES VCR Workflow

7. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it GRIDCC main target areas (Large-scale) scientific experiments High energy particle physics (Radio-) Telescopes Remote process control Accelerator control (Tele-) Biomedicine Robotics Automotive Electronic microscopes Widely Sparse Instrumentation Power Grids Monitoring of the territory Monitoring of the sea Geo-hazard prediction Distributed laboratories Transportation monitoring Sensor network GRIDCC Middleware

8. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Remote Operations of ELETTRA: a feedback via Grid Remote Experts Operator I ESECE IM Accelerator IM Position Monitors IM Correctors See DEMO at the exibition booth 2

9. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Instrument Element Requirements Interactive access to allow control and monitoring Standard interface to the physical devices Fast publishing of the data acquired by instruments Fast information (logs, errors, etc.) publishing to track the behaviour of instruments and possibly solve problems Quality of Service and Advanced Reservation Computing Grid integration  Move data from instruments to the Grid storage resources (Storage Element) and viceversa  Process data produced by instruments using Grid computing resources (Computing Element) I

10. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it The Instrument Element (IE) Instrument Control Grid Access Access to Instruments Data/Info Pubblishing Instrument Element Web Service SRM/GRIDFTP Custom (JMS,..... ) Custom + Plugins I’m here Discovery QoS Advance Reservation

11. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it SOA accessible operations to control and monitor the instruments (via VIGS)  execute a command  get / set attributes / parameters Multi channel data production Instrument Element: multichannel I/O VIGS Commands Status Data Subscribers Grid Interaction SRM VCR Storage Element (SE) Storage Element (SE) Storage Element (SE) Logs, Errors, States, Monitors Instruments IMS Parameters ES VIGSVirtual Instrument Grid Service IMSInformation and Monitor Service SRMStorage Resource Management IE

12. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it VIGS IMS Proxy Control Manager Data Collector Real Instruments Data Flow Control Flow State Flow Error Flow Monitor Flow The term Instrument Element describes a set of services that enable the remote control and monitoring of instruments Input Manager Event Processor FSM Engine Resource Proxy Control Manager IMS Instrument Element Architecture Resource Service IMS Problem Solver Instrument Manager Instrument Element Data Publisher Access Control Manager SRM/SE GridFTP SE IMS Instrument Protocols

13. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Instrument Discovery The discovery of the instruments or of the IEs is an issue when the number of elements is high We can have two use cases:  Quasi static case The number of IEs is well defined In this case a register based discovery mechanism can be used. GRIDCC tesbed is using the LCG BDII (Berkely Database Information Index ) based on LDAP The information collected in the BDII follow a GLUE schema  Dynamic case The number of IEs can change very quickly, they are very simple devices, often with poor hardware support The discovery is just use to know which are the online IEs A new approach has been developed based on Peer to Peer (P2P) protocols See DEMO at the exibition booth 2

14. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Instrument Element: Scalable on embedded systems FPGA PPC 405 Xilink Virtex IV Custom Board Custom Electronics 1 Gbps Ethernet Web Service GridCC IM JavaVM Linux USER INTERFACE Custom Logic Montavista www.montavista.com JamVM http://jamvm.sourceforge.net Standalone Axis IE Instrument Manager

15. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it The Virtual Control Room (VCR) The VCR provides a collaborative environment where users:  Meet and collaborate by means of groupware tools (e.g. chat, shared calendar, logbook, video conference tools)  Search, discover and browse resources such as people, instruments and other eInfrastructure resources (e.g. computing and storage)  Transparently operate with remote instruments using the GRIDCC middleware Infrastructure

16. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Architecture of the VCR User interface Layer providing General and Application Specific tools Services Layer providing low-level functionalities needed by the VCR Web-based portal providing some basic functionalities and support for aggregating components

17. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it VCR in action

18. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Grid in the palm of your hand

19. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it WfMS Architecture

20. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Remote Operations Workflows: OrbitFB

21. BoF RISGE - OGF20 Roberto.Pugliese@elettra.trieste.it Final considerations The GRIDCC project is integrating instruments into the “classic” computational/storage Grids. The Instrument Element allows:  virtualisation of the real instruments  support of a variety of instruments from high end to embedded devices The Virtual Control Room is a groupware tools that support distributed and heterogeneous teams of people in the collaborative access to the extended eInfrastructure The Complex workflows integrating instruments, computational and storage resources can be executed Sustainability Production installations Support new user communities