1. VO Tools and Portals: Summary and Conclusions
Flavia Donno, CERN

2. Session Organization The session was organized in several sub-sessions
Virtual Organization Support:
Dedicated to VO and user support support with a special focus on real experience and open issues.
The Experiment Integration and Support (EIS) Team presented its experience mainly serving HEP but also other VOs such as GEANT4 and UNOSAT.
General VO and User Support provided by the EGEE Global Grid User Support (GGUS).
VO Portals
Portals make Grid technology transparent to users. They provide for a very practical way to have users approach the Grid.
The NICE company presented its product, EnginFrame, on which also Genius is based.
VO Monitoring
Monitoring tools are very needed by operators, middleware experts, users, and VOs to monitor resource usage and correctly tune the Grid.
The INFN Team presented GridICE, present status and future developments.
F. Donno et al, EGEE User Forum, CERN, March 2006

3. Session Organization The session was organized in several sub-sessions
VO Application and Software Management
Among the tools needed to VO communities, the ones to integrate and run legacy applications on the Grid and manage and distribute application software at all sites participating to provide resources to a VO are fundamental.
The P-Grade Portal and the GEMLCA bridge between different Grids to solve Grid interoperability problem at the workflow level and at the same time provide a way of transparently run legacy applications on different Grid infrastructures.
The ETICS project aims to provide a building, testing and certification service for Grid middleware with the possibility of extensions to VO software environment and applications.
F. Donno et al, EGEE User Forum, CERN, March 2006

4. Session Organization The session was organized in several sub-sessions
Other Tools and infrastructures:
This subsession was dedicated to other interesting talks presenting ideas sometime not yet explored in EGEE.
A proposal was presented to introduce Universal Accessibility of the Grid via a Metagrid Infrastructure.
Ideas on a methodology for VO design and management
F. Donno et al, EGEE User Forum, CERN, March 2006

5. VO and User Support EIS : Experiment Integration and Support Team
Help LHC Experiments integrating their production environment with the Grid Middleware and utilities.
Offer support during all steps of integration process
understanding of the middleware functionality
testing new prototypal components
getting on the LCG Infrastructure.
Production is the main focus.
Experiment Integration Support does not mean User Support
(However….Because of the familiar and friendly relationship installed with the experiments very often we really represent the quickest first line support ….).
Experiment Integration Support does not mean GOC nor CIC
(nonetheless we provide customized monitors of the GRID, we manage production grid services and/or we are active in chasing site related problems weekly meeting with Operational).
F. Donno et al, EGEE User Forum, CERN, March 2006

6. VO and User Support Help with middleware functionality and usage.
Integration
Help with middleware functionality and usage.
Propose new grid technologies to the experiments.
Perform functionality tests on new technologies.
Provide customized distributions and missing tools
Discuss requirements and bring them to the attention of the developers
Understand issues and channel them to developers or M/W experts.
Experiment and User Support
Provide documentation: Manuals, Guides, FAQ, tutorials.
Answer first line User Support questions
Monitoring experiment specific production system.
Provide infrastructure expertise
Monitoring/Managing GRID and Experiment Specific Services
Chasing mis-configured sites and solving site-related problems
Service Challenge Second Level Support (on shift)
F. Donno et al, EGEE User Forum, CERN, March 2006

7. VO and User Support
Comments:
The EIS model has been proven to be very incisive and very useful for High Energy Physics communities.
Other VOs have tried to follow the same example, however the shortage of internal human resources has made the entire exercise difficult
The need to have a more wide Grid User Support infrastructure to assist all communities during their integration and exercises with Grid has clearly emerged.
F. Donno et al, EGEE User Forum, CERN, March 2006

8. VO and User Support
A possible answer ?
Global Grid User Support is the support infrastructure for Grid users, deployment and operation problems
It offers a great variety of services to satisfy user needs at all levels
It does not substitute but integrates existing infrastructures and coordinates support efforts
F. Donno et al, EGEE User Forum, CERN, March 2006

9. Central Application (GGUS)
VO and User Support
The support model in EGEE can be captioned
“Regional Support with Central Coordination"
The ROCs and VOs and the other project wide groups such as the Core Infrastructure Center (CIC), middleware groups (JRA), network groups (NA), service groups (SA) are
connected via a central integration platform provided by GGUS.
….. …
Central Application (GGUS)
Deployment
Support
RC 1
RC X
Middleware
Network
Operations Support
TPM
BIOMED
ESR
DS 1
DS 5
MS 1
MS 8
ROC 1
ROC 12
ROC…
VO Support
ALICE
Interface
Webportal
Other Grids
F. Donno et al, EGEE User Forum, CERN, March 2006

10. VO and User Support
GGUS provides a single entry point for reporting problems and dealing with the grid.
In collaboration with the EGEE EIS team, the EGEE User Information Group, NA3, and the entire EGEE infrastructure, GGUS offers a portal where users can find up-to-date documentation, and powerful search engines to find answers to resolved problems and examples.
Common solutions are stored in the GGUS knowledge database and Wiki pages are compiled for frequent or undocumented problems/features.
GGUS offers hot lines for users and supporters and a VRVS chat room to make the entire support infrastructure available on-line to users.
Special tools and grid middleware distributions are made available by the NA4/EIS team for GGUS users.
GGUS is interfaced with other grids’ support infrastructures such as in the case of OSG and NorduGrid.
GGUS is used for daily operations to monitor the grid and keep it healthy. Specific user problems can be directly communicated to the Grid Operation Centers and broadcasted to the entire grid community.
GGUS is used also to follow and track down problems during stress testing
activities such as the HEP experiments production data challenges and the service
challenges.
F. Donno et al, EGEE User Forum, CERN, March 2006

11. VO and User Support
Comments:
The GGUS system offers quite a satisfactory response to user needs.
However VO Dedicated support is missing. More funding is necessary to establish dedicated support groups.
Connection between GGUS and Local Helpdesks in terms of knowledge and statistics seems missing. Even if knowledge is somehow available, statistics are missing. GGUS is working with the metrics group.
There is still little involvement from the VOs. This is due to many reasons: missing human resources, most of the effort for HEP covered by EIS. This is already changing.
F. Donno et al, EGEE User Forum, CERN, March 2006

12. VO Portals
F. Donno et al, EGEE User Forum, CERN, March 2006

13. VO Portals 1.x - LAN Integration (1998)
Unix-Windows integration for simplified parallel and distributed computing
2.x - WAN MetaComputing interface (1999)
Standards convergence, dynamic contents management
3.x - Computing Portal ( )
Web architecture, JS independence, Globus integration
3.5 – Enterprise Grid Portal (2004)
Fault tolerance, Auth delegation, Authorization
4.x – Grid Gateway (2005)
WS-I compliancy, user mapping, enhanced Data management, RSS notification, JSR168 portlet gateway
F. Donno et al, EGEE User Forum, CERN, March 2006

14. VO Portals
F. Donno et al, EGEE User Forum, CERN, March 2006

15. VO Portals
F. Donno et al, EGEE User Forum, CERN, March 2006

16. VO Portals VO Portals Comments:
Are a very practical way to have users approach the Grid. Different portal with different functionalities can and should be integrated.
F. Donno et al, EGEE User Forum, CERN, March 2006

17. VO Monitoring A VO wants to monitor a Grid in order to:
to observe the composition, state and features of available resources available to its users
to analyze their behavior and performance
to track user activity as regards computing/storage/network resource usage
to detect fault situations
In the context of Grid computing, two important categories of monitoring systems are:
Application monitoring
Infrastructure monitoring
We focus on
F. Donno et al, EGEE User Forum, CERN, March 2006

18. VO Monitoring GridICE Server Site – Administrative domain
Lemon or other fabric mon. tools
charts
HTML
XML
notification
data aggregation and abstraction
Grid Inf. Service
persistent storage
discovery
consumers
scheduler
Grid Discovery Service
Site – Administrative domain
Monitored Entity
Site Collector
local publisher
site consumer
site publisher
sensor
site persistent storage
F. Donno et al, EGEE User Forum, CERN, March 2006

19. We focus on the following categories of users:
VO Monitoring
We focus on the following categories of
users:
VO manager
actual set of resources accessible to VO members
Grid operator
all resources under responsibility of a Grid Operator Center
Site administrator
site resources offered to a Grid
F. Donno et al, EGEE User Forum, CERN, March 2006

20. VO Monitoring Comments:
Technical questions about the usage of GridICE. -
The notification service is a very useful one: it should be integrated in the current support system. Developers working on the new notification service. Only specific events are monitored at the moment. In future working on an XML description.Integration in GGUS and CIC portal.
There are more than one tool. Is this OK? In general it is good. EGEE is a service oriented project. There should not be a monopoly of a given service. The same applies to WMS, DM, etc.
F. Donno et al, EGEE User Forum, CERN, March 2006

21. VO Application and Software Management
The P-Grade Portal
General purpose, graphical, workflow-oriented Grid portal
Supports the development and execution of workflow-based Grid applications
Components of the workflows can be
Sequential jobs
Parallel jobs (MPI, PVM)
Legacy code (GEMLCA) services
Enables the exploitation of two levels of parallelism
intra-job
inter-job
Enables to control and observe the execution of the workflow applications
Enables fault-tolerant workflow execution
F. Donno et al, EGEE User Forum, CERN, March 2006

22. VO Application and Software Management
The P-Grade Portal
Enables application-oriented extensions by new portlets
Hides the low-level Grid details
Enables the access of every important feature of the underlying Grid, e.g.:
brokering service
information service, etc.
Easy porting of legacy and other applications to the Grid
Solves the interoperability of various Grids at the workflow level and hence enables:
easy porting of legacy and other applications between various Grids (e.g. from LCG to gLite)
parallel execution of a workflow in several Grids
F. Donno et al, EGEE User Forum, CERN, March 2006

23. VO Application and Software Management
Proxy management
Grid resources management
Workflow creation
Job mapping to Grid resources
Workflow management and
execution visualization
F. Donno et al, EGEE User Forum, CERN, March 2006

24. VO Application and Software Management
ETICS
Vision: A dependable, reliable, stable grid infrastructure requires high-quality, thoroughly tested, interoperable software middleware and applications
Mission: Provide a generic service that other projects can use to efficiently and easily build and test their grid and distributed software. Set up the foundations for a certification process to help increasing the quality and interoperability of such software
F. Donno et al, EGEE User Forum, CERN, March 2006

25. VO Application and Software Management
Web Application
Web Service
Via browser
Report DB
Project DB
Build/Test
Artefacts
NMI Scheduler
Via command-
Line tools
NMI Client
Wrapper
WNs
ETICS Infrastructure
Clients
F. Donno et al, EGEE User Forum, CERN, March 2006

26. VO Application and Software Management
Comments:
There are different and sometimes more complex problems connected with VO software management and distribution. Can ETICS address those ? The ETICS team has stressed that the project focuses on application software management as well.
Important is to establish QA metrics depending on the middleware/software in question. ETICS intends to do so.
P-Grade is an interesting proposal for hiding the details of different flavors of Grids. Based on fat clients.
F. Donno et al, EGEE User Forum, CERN, March 2006

27. Other Tools and infrastructures
UNIVERSAL ACCESSIBILITY OF THE GRID
rooted in the concept of Design for All in Human Computer Interaction
addresses accessibility problems in human interaction with software applications and telematic services.
F. Donno et al, EGEE User Forum, CERN, March 2006

28. Other Tools and infrastructures
Various application domains benefited from research and development in this area including:
interactive television and media
children educational games
interfaces for the physically handicapped
interfaces for various devices
F. Donno et al, EGEE User Forum, CERN, March 2006

29. Other Tools and infrastructures
Methodology for the definition of a VO: VO definition in terms of resources needed, policies to be applied, software environment, human resources, workflow, scheduled events, etc.
VO Modification: Define procedures to add more resources, get feedback from past experience, etc.
Can some of this task be automated ?
F. Donno et al, EGEE User Forum, CERN, March 2006

30. Other Tools and infrastructures
Comments:
The proposal is to extend the concept of interoperability toward universal accessibility.
Universal accessibility has not been taken into account at design time
Is it probably too early to speak about universal accessibility ?
VO methodology definitions should be connected to operations and translated into practical procedures. Connection to support infrastructure is also needed.
F. Donno et al, EGEE User Forum, CERN, March 2006

31. Conclusions Very useful workshop.
Quite some ideas about commonly used tools came out.
Unfortunately VO communities were not really represented.
We look forward to the next EGEE User Forum!.
F. Donno et al, EGEE User Forum, CERN, March 2006