Elettra Virtual Collaboratory: the evolution of a Virtual Laboratory Software from a simple web application to the GRIDCC Roberto Pugliese, Alessandro Busato, Alessio Curri, Enrico Mariotti, Daniele Favretto, Fulvio Billè, Roberto Borghes, Fabio Asnicar, Valentina Chenda, Laura Del Cano, Lawrence Iviani, Michele Turcinovich and the GRIDCC collaboration Sincrotrone Trieste – ELETTRA Instruments and Sensors on the Grid IEEE Conference on e-Science and Grid Computing Melbourne, Australia, on 5-8 December 2005

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Outline The Elettra Virtual Collaboratory (EVC)  Work The BIOXHIT project  Virtual Collaborative System (VCS) The EUROTeV project  The Global Accelerator Network  The Multipurpose Virtual Laboratory (MVL) Evolving EVC to meet VCS and MVL requirements The GRIDCC project  The Multipurpose Collaborative Environment (MCE) Evolving EVC with MCE

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese What is a Collaboratory? The term “collaboratory” was coined by William Wulf by merging the words collaboration and laboratory, and defined as “... Center without walls, in which researchers can perform their research without regard to geographical location - interacting with colleagues, accessing instrumentation, sharing data and computational resource, and accessing information in digital libraries”.

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese What is a Collaboratory? In particular, the core capabilities that constitute a collaboratory can be seen as technologies to link:  People to people (e.g., electronic mail, and tools for data conferencing, such as VRVS)  People to information (e.g., the World Wide Web and digital libraries)  People to facilities (e.g., status of remote instruments) to enhance utilization by expanding access to resources

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese What is the Elettra Virtual Collaboratory (EVC)? EVC is an example of virtual laboratory, a system which allows a team of researchers distributed anywhere in the world to perform a complete experiment on the equipped beamlines and experimental stations of Elettra. User at ELETTRA Team Member at Home Lab Remote Collaborator Equipment Control Data CPU COLLABORATORY

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese What is Elettra?

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese EVC usage scenarios: Cristallography “by mail” EVC allows biologists to send by mail protein crystals which will be ananlized at the Xray Diffraction beamline by the beamline staff. Collected data and results are accessible via EVC and results can be downloaded as soon they are available.

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese EVC in action: a web portal EVC is based on the “web portal” metaphor All you need is a browser EVC supports four different user categories:  Visitors  Normal users  Project leaders  Staff

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Collaborating to an EVC project Scientists working to an EVC project can use many project related collaboration tools EVC presents an adaptive interface changing to suite the category and expertise level of the user

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Collaboration Tools: EVC chat EVC chat is “project centered”: there is a different channel for each project Usual chat feature are extended in order to allow exchange of  Drawings  scientific images  graphical annotations

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Collaboration tools:scientific visualisation Scientists can browse, visualise and process remotely scientific data in real-time as soon as the data is collected

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Collaboration Tools: telepresence The different video streams of the equipped experimental stations can be selected and viewed even through a slow connection Movable cameras can be controlled via web by the project leader

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Collaboration tools remote computing Legacy software is normally not web enabled EVC uses VNC to web enable legacy apps.  It is small and simple, sharable and open  Can be tunnelled via ssh VNC can be used as a fast integration tool

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Collaboration Tools: Remote Beamline Control and Supervision Beamwatch presents a synoptic view of the beamlines Autorised people can thus operate remotely on the beamline intrumentation

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese EVC Architecture EVC has 2 main components:  the application server and  a set of nodes The application server is running the portal application, the user and project database; the application server activates actions implemented by agents running in the nodes or requests services to external systems node1node2 nodek-1 nodek application server DB

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese EVC facts EVC project started on June 2001 and finished on June The first prototype was installed on the Xray Diffraction beamline of ELETTRA on June 2002 EVC is now operating on all the beamline and experimental station of ELETTRA and acts as the web interface to the Elettra Scientific Computing Environment (instruments, computing farms, storage) EVC was presented at SMAU2002, NOBUGS2002 and SMAU2003, HCI2003, NOBUGS2004 EVC development staff is partecipating in many EU founded projects under FP6 (BIOXHIT, IA-SFS/JRA1, GRIDCC, EUROTeV/GAN)

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese BIOXHIT Virtual Collaboratory System The BIOXHIT project which will develop an integrated platform for high-throughput structure determination ELETTRA is developing the Virtual Collaboratory System a Virtual Organization (VO) connecting all the European laboratories doing research in the field of structural genomics.

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese BIOXHIT VCS scenario Supporting Services VO user crystallisation beamline 1 beamline 2 beamline 3 Processing Farm Processing Farm Data Storage VCS will be used to implement a widley distributed Virtual Organisation (VO) connecting all the stations and Laboratories involved in the BIOXHIT project

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese VCS EMBL-Hamburg

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese EUROTeV/GANMVL In the EUROTeV project the design study of the International Linear Collider ELETTRA is developing the Multipurpose Virtual Laboratory, the core tool to implement the Global Accelerator Network, a VO connecting all the international laboratories doing research in the field of Accelerators. Remote control of an accelerator facility has the potential of revolutionizing the mode of operation and the degree of exploitation of large experimental physics facilities. The first prototype of the system planned by April 2005 will allow the remote control of ELETTRA storage ring from DESY.

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Work

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Work

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese VCS/MVL Virtual Organisation

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Evolving EVC to implement VCS / MVL In this architecture every institute in the VO should have a VCS Node. The node can support more stations (e.g. a data collection station, a cristallisation, a control room station, a movable station etc). Stations can share resources and tools. The remote collaborator will use his PC equipped with a web browser and if the case with a projector. All the communication (AS-to-AS and AS-to-LN) is done via webservices secured with X.509 certificates by mutual authentication Legacy applications are integrated using VNC if they do not have a web interface or ssh tunnels and proxy or redirection if they already have a web interface.

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Evolving EVC to implement VCS / MVL All the communication between the Application Servers located in the distributed laboratories is done via webservices (Axis implementation). All the Local Nodes run a Local Node Server. Comunication between the AS and the LN is done via webservices (gSOAP implementation). The systems are also equipped by a Management Station which allows easy configuration and maintenance via web browser. VCS and can be considered a sort of integration platform. Scripts stored in the AS database are transferred to the local nodes, executed and the results returned to the user via the AS.

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese The GridCC Project Instruments GridComputational Grid +

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese GRIDCC project in 3 steps Development of generic Grid middleware, based on existing building blocks (Grid Services) which will allow the remote control and monitoring instrumentation such as distributed systems. Testing of the middleware on challenging applications to validate it both in terms of functionality and quality of service:  European Power Grid  Geo-hazards  Remote Operation of an Accelerator Facility  High Energy Physics Experiment  … Dissemination of the new software technology to encourage a wide range of enterprises to evaluate and adopt our Grid-oriented approach to real-time control and monitoring of remote instrumentation.

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese The MCE and the GRIDCC landscape Supporting Services Virtual Ctrl. Room Virtual Ctrl. Room Diagnostics Instrument 1 Instrument 2 Instrument 3 The MCE is a software to implement Virtual Control Rooms, i.e., multi-user, collaboration-supporting interfaces to a widely distributed control system with access to grid-enabled computing and data storage facilities StorageElement ComputingElement

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese GridCC Architecture Information Monitor Service Computing Element Computing Element Computing Element Storage Elements Storage Elements Storage Element Global Problem Solver Virtual Control Room Virtual Control Room Security Service AutS TGS PolR Instrument Element Instrument Element Instrument Element Collaborative Service Exec. Service WfMS WMS AgrS

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese The Multipurpose Collaborative Environment a groupware providing general purpose services and interfaces to support collaboration among researchers and operators, to control remote instrumentation, and other tasks related with experimental activities will be used to implement the VCR for the different GRIDCC pilot applications through customization and integration with application-specific services. based on a core groupware application (providing, e.g., authentication, management of the VO users and instruments) and a set of plug-ins:  General purpose (e.g., chat, notebook, video conference)  Specific to the particular application (e.g., accelerator control, specific instrument control)

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Design and Development Approach State of the art evaluation  Collaboration support tools for scientific experimental activities  Available technologies Derivation of general requirements  through use cases, interviews, exchanges with other related projects (e.g., EUROTeV) Prototyping  Discussions over interface sketches  Incremental development of functional prototypes

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Design Choices Web-based portal interface  no installation, porting, clients available almost anywhere  Minimal system requirements: Web browser, JRE for applets Exploit portlet technology  Why? manageable integration of application-specific functionalities within the MCE  Current prototypes are based on a modified version of the GridSphere framework, developed under the GridLab EU project  Collaboration with the GridSphere team to extend and improve the framework to our needs was established and some of our developments are already in GridSphere codebase

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Login page

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Personalisation

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese eLogbook

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese VO view: people browser and chat

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Resource browser and Instrument control

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese More Instruments Control

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Video/Phone Conference: VRVS/Skype

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Desktop Sharing

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Execution Services: job submission

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Execution Services: File Access

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese Evolving EVC with MCE EVC project started in 2001 and has now more than 4 years of operations EVC software has been improved in these years moving from a simple single facility web application to a multi-facility integration platform based on webservices We are currently refactoring EVC in order to migrate to the GRIDCC MCE middleware.

Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese GRIDCC Architecture IE VCR IE ExeS IE CE IE SE IMS SecS PS VCR WfMS WMS AgrS AutS TGS PolR VIGS ACM IMDM IMSpx RS LPS CollabS