1. NA3 – User Community Coordination
Steve Brewer
Chief Community Officer
EGI.eu
NA3 – EGI-InSPIRE EC Review 2011

2. NA3 – EGI-InSPIRE EC Review 2011
NA3 Overview
Spain
Greece
Portugal UK
Slovenia
Czech Republic
France
Finland
Poland
Russia
Georgia
Armenia
Italy
Serbia
Israel
Germany
Hungary
Netherlands
Moldova
Norway
Switzerland
Ireland
Turkey
Denmark
Cyprus
Slovakia
Albania
Sweden
Lithuania
Taiwan
Philippines
Indonesia
Korea
Singapore
Malaysia
Thailand WP
Beneficiary
Total PMs
WP3-E
EGI.eu
201
CSIC 12
GRNET 47
LIP
STFC 23
WP3-N
ARNES 19
CESNET 28
CNRS 54
CSC 24 27
CYFRONET
34.9
E-ARENA 7
GRENA 6
IIAP NAS RA
INFN 40
IPB 16
IUCC 13
KIT-G 66
MTA KFKI 32
NCF 9
RENAM
SIGMA
SWITCH 8
TCD 14
TUBITAK 36 WP
Beneficiary
Total PMs
WP3-N
UCPH 20
UCY 8
UI SAV 38
UPT 24
VR-SNIC 9 VU 15
ASGC
121
ASTI
220
ITB 10
KISTI 28
NUS
UPM 12
NECTEC 4
36 Countries
37 Beneficiaries
1379 PMs
28.71 FTEs
NA3 – EGI-InSPIRE EC Review 2011

3. NA3: User Community Coordination
Objectives
Activity
Achievements and challenges
Plans and summary
NA3 – EGI-InSPIRE EC Review 2011

4. 1) Objectives: coordinating the user experience
VRCs communicate with EGI through the User Community Board
User Community Support team
(NA3.2)
VRC with many research groups (and one VRC representative)
EGI dissemination
Research group
National and specialist support & services
NGI User Support coordination
(NA3.3)
Support teams available across 50 partners
Researchers and scientists
Application
developers
EGI User Community
Support Services
(NA3.4)
The task we were given:
To improve the EGI user experience and user services through:
Coordination of the management of support service from national and specialist support units
Coordination of user-related requirements relating to support services and training
Coordination of technical services to support the establishment and management of virtual organisations (VOs)
TNA3.1: Activity Management
Steve Brewer, EGI.eu, 3% of total NA3 effort
TNA3.2: User Community Support Team
Steve Brewer, EGI.eu, 30%
TNA3.3: NGI User Support Teams
50 partners, Gergely Sipos, EGI.eu, 49%
TNA3.4: Technical Services
Marios Chatziangelou, GRNET, 18%
The UCB is the formal communication channel between VRCs and EGI.
Local System administrators
Research leader
Application Database, Training Marketplace, VO Services, Requirements Tracker
Different user types collaborate within a research group
NA3 – EGI-InSPIRE EC Review 2011

5. NA3: User Community Coordination
2) Activity
NA3 – EGI-InSPIRE EC Review 2011

6. Integrate current and future use
Activity Management (NA3.1)
User Support Coordination (NA3.2)
User Community Support (UCS) team of EGI.eu
Ensures the flow of information, knowledge and solutions among stakeholders
Support teams
NGI User Support Teams (NA3.3)
Experts within established user communities
Site administrators
Projects with focus on specific areas. e.g. EDGI; MAPPER; StratusLab
Users
Teams
Processes
Services
Processes (NA3.2)
User Community Board (UCB)
VRC accreditation
Requirements processing
Training Working Group
User Services Advisory Group
Technical services (NA3.4)
Application Database
VO services
Training Marketplace
Requirements Tracker
This slides provides further detail on the Tasks within the Work Package from the end-user’s perspective
NA3 – EGI-InSPIRE EC Review 2011

7. VOs and VRCs within the EGI Ecosystem
User Community
A VRC can be a group of large-scale research collaborations, or a number of separate VOs grouped according to research domain
A VO is a group of people (e.g. scientists, researchers) with common interests and requirements, who need to work collaboratively and/or share resources
Virtual Research Communities communicate with EGI through the User Community Board
Requirements + Feedback
Services + Support
UCB
Policies + Funding
Service & Resource Providers
EGI-InSPIRE
Requirements + Feedback
Public Funding Bodies
Policies + Funding
Technology Providers
European Commission
EGI.eu
This slide provides further information about the context of where VOs and VRCs sit within the EGI eco system
The UCB comprises representatives of the user communities and met regularly to review and plan user support coordination.
Open Source Providers
Resource Infrastructure Providers
National Research Councils
Strategic Feedback
Technology + Support
Commercial Providers
Policies + Funding
NA3 – EGI-InSPIRE EC Review 2011

8. UCB and the Virtual Research Community model: purpose and benefits
UCB: provides channel between VRCs and EGI
Benefits:
Defined communication channels: named contact points between communities & EGI
Application integration
Policy and procedures: define interaction between EGI and users
Requirements gathering: solutions and influence
Dissemination: sharing of news, breakthroughs, best practice and innovation
User Community Board comprised representation from: Life Sciences, WLCG, WeNMR, Astronomy & Astrophysics, Earth Sciences, Computational Chemistry, Arts & Humanities, Hydrometeorology
NA3 – EGI-InSPIRE EC Review 2011

9. A virtuous service cycle: current communities
Used by Researchers
Gathering New Requirements
New Technology Assessed
Deployed Infrastructure Services
EGI.eu
National
Resource Providers
New User Communities
European Grid Infrastructure
Ecosystem
MoUs & OLAs
Technology Providers
MoUs & SLAs
Current User Communities
MoUs
Current Use Communities include: Life Sciences, WLCG, Astronomy & Astrophysics, Fusion and Earth Sciences (MoUs proceeding as legal (or project) entities to represent the VRC are confirmed
New User Communities include: Arts, Humanities and Linguistics, Hydrometeorology, Seismology, Structural Biology…
New requirements are discussed and evaluated during the VRC accreditation process – all part of the negotiation
NA3 – EGI-InSPIRE EC Review 2011

10. NGI User Support Teams (Task NA3.3)
NGI User Support Teams provide:
Consultancy & training
Access to grid services
Application porting
Developing and operating software services
Collecting feedback
Documentation & helpdesk
NGIs receive:
Web & wiki resources
Requirements process
Applications and services
VRC coordination support
NGI User Support Teams
Consultancy
Training
Provide access to grid and software services
Porting apps.
Dev. and operate software services
Collecting Feedback
Document
Processes & apps.
Helpdesk
Serbian NGI: Grid Intro March 2011
SIMAP added: 4 March 2011, (Comp. Chem)
Support teams are available across 50 partners – they are all different… as are their support skills.
NA3 – EGI-InSPIRE EC Review 2011

11. A virtuous service cycle: new communities
National
Resource Providers
European Grid Infrastructure
Ecosystem
Current User Communities
Used by Researchers
Gathering New Requirements
New Technology Assessed
Deployed Infrastructure Services
MoUs & OLAs
MoUs
EGI.eu
New User Communities include: Arts, Humanities and Linguistics, Hydrometeorology, Seismology, Structural Biology…
New requirements are discussed and evaluated during the VRC accreditation process – all part of the negotiation
New User Communities
MoUs & SLAs
Technology Providers
NA3 – EGI-InSPIRE EC Review 2011

12. New user communities: gathering new requirements
Integrated Services
Human
Technical
Infrastructure
What makes up a Virtual Research Community?
VRCs provide EGI with:
EGI offers VRCs:
Virtual Organisations
Mailing lists
Workshops
Forums
Blogs
Projects
Shared stories
Best practices
Collaborations
Laboratories
Partnerships
Requirements
Applications and tools
Data collections
Training modules
Success stories
Experts
Users
Easy to use infrastructure
Grid-ready applications
Training resources
Help and support
Involvement in open development processes
Workshops and forums
Design
Deliver
Discover
How do we search for and identify new communities?
Service can become sustainable if they are what is needed:
- demand creates consensus and participation
This is a cyclical process:
- user-driven development; community-motivated delivery
- keep learning, keep improving efficiency of evolution process
NA3 – EGI-InSPIRE EC Review 2011

13. NA3 – EGI-InSPIRE EC Review 2011
NA3 – EGI-InSPIRE EC Review 2011

14. Task NA3.4: Technical Services
Provided by a few NGIs for the whole community
To support the Virtual Research Communities
To facilitate the work of NGI support teams
To improve communication among NGIs, among users and with users
Tools and services (Task leader: Marios Chatziangelou, GRNET):
EGI Application Database (Marios Chatziangelou, GRNET)
Domain specific application – for scientific end users
Reusable toolkits and services – for NGI support teams
Services for VOs (Ignacio Blanquer Espert, CSIC; Gonçalo Borges, LIP)
Tools, manuals, assistance for VO setup and management
Training Marketplace (Claire Devereux, STFC)
Repository of events and materials
Place to request and provide NGI training services
Requirement Tracker (EGI.eu with help from CESNET)
To capture needs from the whole community
To offer solutions from NGIs
These are taken from the DoW. Training has been revised to better fit purpose.
Note that Greece’s effort is equal to Ibergrid and UK combined.
NA3 – EGI-InSPIRE EC Review 2011

15. EGI Application Database
Benefits of AppDB:
gives recognition to reusable applications and tools
gives recognition to application developers
Use of AppDB
Register applications/tools
Reuse of applications/tools
Map of application landscape
AppDB gadget has been instantiated by:
WeNMR, LSGC, GridPP, Serbian, UK and Hungarian NGIs
NA3 – EGI-InSPIRE EC Review 2011

16. NA3 – EGI-InSPIRE EC Review 2011
AppDB: PY1 statistics
New additions to the Applications Database during Project Year 1
(Data: largest to smallest)
Note: Apps. Can be associated with > 1 country
NA3 – EGI-InSPIRE EC Review 2011

17. NA3 – EGI-InSPIRE EC Review 2011
Training marketplace
Integration of:
Training event calendar
Digital library (of training materials)
Service offering & requesting form
Purpose of Training Marketplace:
Register training events
Use, reuse and share training materials
Offer training-related services (infrastructure, VO, CA, ...)
Feed training-requirements
Delegate representative to EGI Training Working Group
Results:
Approximately 60 training events were added during Project Year 1.
Training Marketplace launched: input from Training Working Group new features & legacy data from Digital Library and events calendar now available (~60 training events were added during PY1.
NA3 – EGI-InSPIRE EC Review 2011

18. Training Marketplace: Motivation
The Training Marketplace was devised to overcome the failings of the University of Edinburgh (UE) to deliver the Training Events and Training Repository as part of the UK JRU led by STFC:
UE produce no significant new functionality or project plans in PQ1 & 2
In January 2010 UK JRU advised EGI that Edinburgh were withdrawing and that effort would be reallocated within the UK.
Training Working Group requirements document ( drives development of Training Marketplace model
STFC takes on responsibility of training services in TNA3.4
New technology (Drupal) used importing old training material as required
NA3 – EGI-InSPIRE EC Review 2011

19. NA3 – EGI-InSPIRE EC Review 2011
Services for VOs
Activities within the sub-task:
Consultancy and helpdesk for VO managers
Research and evaluation of VO management, monitor and accounting tools:
NAGIOS SAM, GANGA, DIANE, CERN dashboard & mini dashboard
Specification of requirements for operational tools to simplify VO management
Provision of VO support software for VRCs and Vos – in use by:
WeNMR, BioMed (LifeSciences), phys.vo.ibergrid.eu, life.vo.ibergrid.eu and ict.vo.ibergrid.eu
NGI VO
monitor
NA3 – EGI-InSPIRE EC Review 2011

20. VO Services: NAGIOS SAM
NAGIOS SAM triggers the executions of probes in grid sites
Tests to the Storage resources
Tests to the Computing Resources
Summary views of successful and unsuccessful services
Sends s to the site admins in case of failures
Benefits:
Sites from a VO are tested (infrastructure tests run probes on NGIs)
Ability to develop and integrate VO’s own probes
A VO can use the central installation from LIP/UPV, or can install its own
Service Status Details for: phys.vo.ibergrid.eu
NA3 – EGI-InSPIRE EC Review 2011

21. Processing EGI requirements; implementing solutions
TCB
EGI Requirements Tracker
VRCs
Input Channels for User requirements
NGIs
HUCs
ESFRIs
EEF
VOs
Developer teams in TNA3.4
UCST (with NGIs)
UCB
External technology providers
EIROs
EGI Helpdesk (GGUS system)
EMI
IGE
SAGA,
...
110 18
22 (+ 60) 30 4 20 36
60 requirements captured from documents (have no “owner” from the community yet)
Red figures: open requirements
Green figures: resolved requirements
NA3 – EGI-InSPIRE EC Review 2011

22. Requirements Tracker (RT)
Permanent storage for EGI requirements
User
Operational
Complete transparency
For user communities
For NGIs
For technology providers
Requirement submission interface
Query interfaces
Manuals available
NA3 – EGI-InSPIRE EC Review 2011

23. PY1 Results: Requirements Tracker
110 new requirements:
48 resolved within EGI-InSPIRE/UCST
36 prioritised by UCB of which…
20 passed via TCB to Technology Providers
26 still being investigated by UCST
Current issues endorsed by UCB and recognised by TCB:
Increased stability and scalability for gLite WMS
Better (more verbose and informative) error messages
Fixing the known bugs before adding new features
Coherency of command line commands, parameters and APIs
Better feedback about jobs, automated resubmission of jobs stuck on sites
Requirements to be refined in community workshops
Data management, Workflow systems & Workload management
Use to define and build EGI VRC User support platform
Proved to be an effective communication channel with EMI and other external Technology Providers
60 requirements also generated from legacy documents from VRCs:
SHARE Roadmap (HealthGrid), ESFRI project reqs., Production Grid Infrastucture – use Case Collection (OGF), Request for Enhancements of gLite to support bio-NMR applications, gLite requirements collected by the EGEE Project Technical Forum, User requirements (EGI-DS), DRIHMS Questionnaire results (D2.2, D3.2)
NA3 – EGI-InSPIRE EC Review 2011

24. NA3: User Community Coordination
3) Achievements and challenges
NA3 – EGI-InSPIRE EC Review 2011

25. NA3 – EGI-InSPIRE EC Review 2011
PY1: Achievements
VRC model developed and deployed:
Accreditation process and MoU template created
MoUs Signed: Structural Biology (WeNMR) & nearly for Life Sciences (LSGC)
MoUs Discussions: WLCG, Hydrometeorology, Fusion, Astronomy & Astrophysics, Comp. Chem.
LoI Discussions: Humanities (DARIAH/CLARIN)
EGI User Community bodies established:
User Community Board established (2 meetings in PY1)
Processes established now meeting monthly
HUCs and new communities participate;
Requirements process developed with TCB
User Services Advisory Group (2 meetings in PY1)
USAG coordinates technology development teams
Contributes to development of Support Service Platform
Mention: UCS support team established in Amsterdam
AppDB – Applications; tool is framework for developing applications eg – PGRADE, infrastructure services from the community
NA3 – EGI-InSPIRE EC Review 2011

26. NA3 – EGI-InSPIRE EC Review 2011
Impact of NA3
NGI contacts established:
Communication building with & within partners
Dedicated EGI User Support booth at UF
UCST related workshops at UF & TF
40-50 attendees
Generated new requirements
Clarification of existing requirements
Dissemination through articles & interviews:
iSGTW -
NGS Newsletter -
NA3 – EGI-InSPIRE EC Review 2011

27. NA3 – EGI-InSPIRE EC Review 2011
Resolved issues
Establishing NGI communication channels took a long time
Development of the technical services did not run as planned
Training Services
UE’s withdrawal from UK JRU mean reallocation of work
STFC took over responsibility by end of PY1
VO Services
SA3 VO dashboard tied to Oracle database (ORACLE).
Alternative system implemented using NAGIOS
Now used by WeNMR, LSGC and a number of NGIs
SA3 now also provided mini-dashboards
NA3 – EGI-InSPIRE EC Review 2011

28. NA3 – EGI-InSPIRE EC Review 2011
Unresolved issues
Lack of funding for communities to develop VRCs
Specialist Support Centres were not funded for HUCs
Fewer VRCs were created than envisaged
Working to resolve perceived obstacles for new VRCs
e.g. Astro using wiki and mailing list
Limited contact with ESFRIs
Not all in a state to integrate resources with EGI yet
Working with ESFRIs through LoI model
Adopted by DARIAH and CLARIN
Pending establishment of ERICs as legal basis for VRC
NA3 – EGI-InSPIRE EC Review 2011

29. NA3 – EGI-InSPIRE EC Review 2011
Use of Resources
WP3-E - WP3 (NA3) - NA3 User Community (EGI)
Partner
Worked PM Funded
Committed PM
Achieved PM %
1-EGI.EU
39.8
50.3
79%
12A-CSIC
4.6
3.0
152%
29-LIP
99%
16A-GRNET
8.1
8.5
96%
16E-IASA
6.5
3.3
200%
34A-STFC
5.6
5.8
97%
Total:
67.5
73.8
92%
WP3-N - WP3 (NA3) - NA3 User Community
Worked PM Funded
Committed PM
Achieved PM %
Total:
137.9
166.2
83%
NA3 – EGI-InSPIRE EC Review 2011

30. NA3: User Community Coordination
4) Plans and summary
NA3 – EGI-InSPIRE EC Review 2011

31. NA3 – EGI-InSPIRE EC Review 2011
Plans for next year: NA3.2
UCB meetings with increased frequency
Monthly to generate more interaction with & between VRCs
More asynchronous interaction & communication
Identify more VRCs
Through NGIs and attendance at research community events
e.g. Conferences, workshops…
Identify and support sub-groups within communities
e.g. Seismology, Hydrometeorology…
Series of Workshops planned
Themes taken from requirements analysis:
Aim: Define needs and gaps in order to find sustainable solutions for end-users
NA3 – EGI-InSPIRE EC Review 2011

32. NA3 – EGI-InSPIRE EC Review 2011
Plans for next year: NA3.3
Communication: Promote core activities to NGIs by monitoring usage:
Application Database
Requirement Tracking system
Training Marketplace
User Support Webpage and the EGI blog
The ‘NGI Roadshow’: Help EGI to promote NGIs’ services and to inform NGIs about the tools that exist for them
HealthGrid conference in UK is an early prototype, as is,
DCI Summer School in Hungary
Roadshow model will be presented to NGIs at TF11
NA3 – EGI-InSPIRE EC Review 2011

33. NA3 – EGI-InSPIRE EC Review 2011
Plans for next year: NA3.4
Develop and promote integrated User Support Platform:
Applications Database (AppDB)
Improve quality and reliability of information
Improve search & browse capabilities; notifications
Architecture; cross-browser compatibility
Training Marketplace
Map view of data
Training wish list; advertise expertise & resources
Promotion of teaching modules and courses
Integration of legacy data; advanced search
VO Services
Continue to run existing services
Introduce VO Admin dashboard
More applications & tools will be added to platform when identified
NA3 – EGI-InSPIRE EC Review 2011

34. NA3 – EGI-InSPIRE EC Review 2011
Summary
VRC model implemented
User Community Board established
User Services Advisory Group established
Requirements Tracker established
Technical Services:
User Support Platform based around gadgets
AppDB gadgets: Being used by VRCs and NGIs
UE training services reallocated to STFC
NA3 – EGI-InSPIRE EC Review 2011