1. www.egi.eu EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 User feedbacks and requirements Gergely Sipos EGI.eu User Community Support Team gergely.sipos@egi.eu

2. www.egi.eu Outline The process The captured requirements

3. www.egi.eu The context The evolution of the European Grid Infrastructure must be driven by the users NA3 is responsible for defining and running support processes for user communities –Requirement gathering is a support activity –Requirement management and captured requirements are detailed in this EGI Wiki page and in MS305thisMS305

4. www.egi.eu Goals, requirements for the process Threshold for communities to provide feedback must be low –Useful feedback must be separated from harmful input and noise –Use VRCs as the main contact points to gather requirements Input provided by the communities in different form and with different levels of details will be normalised. –No input is lost or distorted during the processing phase The prioritisation mechanism (especially across user communities) is fair –VRCs can prioritise their own requirements –UCB can prioritise requirements across communities Prioritised requirements are communicated to the right stakeholders of the EGI ecosystem –Use TCB (Technology Management Board) to identify and engage with technology providers The whole process is efficient, i.e. Implementations become timely available on the infrastructure The whole process is transparent –The captured requirements are visible –Progress with requirements is visible –Technology providers can join in

5. www.egi.eu The process TCB (SA2) End users User community coordination (NA3) Operations (SA1) UMD roadmap EMI EMI roadmap Update every 6 months Update every 12 months Update every 3 months Technical services for users (TNA3.4) Operational tools (JRA1, SA1) New/ improved services IGE IGE roadmap Service operators New / improved services Other technology providers Direct input from users More details: MS305 - User Feedback and Recommendations https://wiki.egi.eu/wiki/Requirements_gathering_details D5.1 – UMD Roadmap

6. www.egi.eu TCB EGI Request Tracker The implementation VRCs Input Channels for User requirements NGIs HUCs ESFRIs EEF VOs Normalisation, categorisation Define workplans InSPIRE activities and tool developers UCST Discuss workplans User Services Advisory Group Endorsed and weighted Categorised requirements; UCB External technology providers Categorised requirements; Workplans EIROs

7. www.egi.eu Categorisation of requirements Functional requirement: –Demand for a new feature in a software WMS should identify and resubmit stuck jobs –Demand for a new type of activity User forum should include a session on Application porting tools Non-functional requirement: –Robustness, scalability, interoperability, harmonised,... –Cannot be interpreted by technologist unless it is connected to one/more feature of a service/activity WMS should be more robust Training in EMI must be harmonised with PRACE Endorsed user requirements must have functional aspect

8. www.egi.eu Categorisation of requirement tickets in RT Support service Applications Database Training Services VO Services Web site Gap Support action Training Consultancy Other UMD Compute File transfer Accounting.... Operations Operational Tools Operations Portal GGUS GOCDB Other Tools and applications Domain specific applications Generic Tools Gap Non-functional requirements will be attached to tickets as tags: stability, scalability, interoperability,... Functional requirements are expressed by categorisation of tickets Categorisation of RT tickets

9. www.egi.eu EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 The captured requirements

10. www.egi.eu What happened so far TimeEventOutcomeNo. of captured requirements August- September NGI survey NGI User Support Team status reports Suggestions for the improvements of user support activities 7 (all about NA3 services) August- October VO and HUC survey Feedback on gLite transition from v3.1 to v3.2 Expected enhancements for first EMI middleware release (due in April 2011) 22 (all about middleware enhancement) In September First TCB meeting Agreement on requirement gathering process between NA3 and other activities and projects From October Collecting documents with community requirements Identification of additional community requirements 8 - SHARE report and GGUS tickets 9 - EEF report 13 - OGF Production Grid document 5 - eNMR report 5 - from 5 papers on Training and Education NovemberInforming EGI community about req. gathering process Email sent to HUCs, VOs, NGI User Support Teams, InSPIRE NA3 members about Requirement gathering process, channels 24 November First USAG meeting Workplan presented about Training Services; Application Database; VO services; GGUS; Operation Portal

11. www.egi.eu Captured requirements 3 different categories were identified: –UMD enhancement (ARC, gLite, UNICORE) –Suggestions for User Support activities (support actions; support tools) –Community requirements

12. www.egi.eu UMD enhancements: Overview UMD enhancements –Outcome of survey for VOs and HUCs –Responses from: ~11% of VOs (16% of users) + HEP and Life sciences HUCs –Mixture of functional and non-functional requirements Non-functional requirements are connected to service features –Intention is to forward them to TCB as they are

13. www.egi.eu UMD enhancements: VOs Top requests from VOs: 1.Stability and scalability of components, especially for gLite WMS 2.Better error messages 3.Fix the known bugs before adding new features 4.Coherency of command line commands and parameters 5.Better feedback about jobs, automated resubmission of jobs that are stuck on sites

14. www.egi.eu UMD enhancements HUCs HEP requests: 1.Timescale of the first EMI release is not convenient for the LHC running schedule in 2011 2.There are supposed to be direct channels between WLCG and EMI Life sciences requests: 1.Coherent/homogeneous set of APIs for all middleware services (WS and java), that can be installed on a client without a complete UI (ie an non-SL host) 2.No single-point-of-failure services (VOMS and LFC currently are critical and single point of failures), i.e. redundancy of critical services is needed. 3.Establish strong site decommissioning procedures with support tools – e.g. LFC administration tools: including automatic replication/migration of files if possible and automatic notification of file owners in all cases. 4.WMS workload management: avoid overloading / time-out of data access that cause many job failures (these should be postponed and retried until success) 5.Management of many small files and improved replica selection. 6.Middleware support for pilot jobs 7.Single sign-on through federated identity mechanism to avoid exposing users to X509 certificates 8.SEs publishing their status (downtime, decommissioning, etc) using the GlueSEStatus BDII attribute. That would allow VOs to efficiently process this information with lcg-infosites. 9.grid-mapfile of dCache is cleared from all VO entries when the VO's VOMS server is not accessible. Old gridmap should be kept in such cases

15. www.egi.eu Suggestions for User Support From NGI User Support Teams From Training & Education documents Can be handled within and by NA3 –Most of them are simple to answer Point to a Wiki page, milestone,... –Others are related to User tools Influenced the workplans of these tools

16. www.egi.eu Community requirements: Captured by EEF EEF report provides 9 common requirements fromEEF report –28 ESFRI projects Requirements (just the names): –Consistent identity management with single sign-on –Virtual Organisations –Persistent storage –User support –Training and consultancy –Web-service interfaces –Workflows –Global scope –Integration with cloud systems and volunteer desk-top systems All the EFF requirements are standardisation issues across DCIs: –In their current form they are non-functional requirements for EGI –Must be further detailed before could be endorsed

17. www.egi.eu Community requirements: Captured from SHARE roadmap SHARE roadmap provides joint requirements fromSHARE roadmap –Pharmaceutical ; Epidemiology; Virtual Physiological Human (VPH) communities Topics that emerged and are not in EEF: –Quality of services (guaranteed services) –Robustness, scalability of services –Easy access for end users (SSO is one aspect of this) –Security during every stage of data lifetime Captured requirements: –Flexible, secure, scalable, robust services –Data security, semantic annotation, integration –Quality of services (guarantees for users) –Simple access for end users to ported applications –Persistent storage of data –Urges interaction between VPH researchers and grid developers/providers SHARE requirements must be detailed further to have functional aspect

18. www.egi.eu Community requirements: Captured from eNMR documents eNMR deliverables provide requirements for gLite Topics that emerged and are not in EEF: –Easy access for end users (primarily to data and databases) = SHARE –Standardised APIs –Reliability of grid middleware services = SHARE –Quality of service = SHARE –Security during every stage of data lifetime = SHARE –MPI support

19. www.egi.eu Community requirements: Captured from OGF document Use Case Collection v1 by OGF Production Grid Infrastructure WGUse Case Collection v1 –Written by providers and users of production grids (EGEE, EDGI, Nordugrid, GENESIS, IGE, DEISA, GIN) –Implementations already exists, or are under development –Should be used by developers of standards to guarantee backward compatibility. Not relevant for UCB

20. www.egi.eu Commonalities in community requirements 1.Easy access for end users (SSO is one aspect of it) 2.Reliability of grid middleware services 3.Guarantees for Quality of Service 4.Secured management of data (during every stage)

21. www.egi.eu Requirement gathering and processing What’s new? Online form to provide feedback to EGI User Community Support: –http://www.egi.eu/user-support/provide_feedback/http://www.egi.eu/user-support/provide_feedback/ Configuring RT for requirement management has started UCST initiates workshops to capture and further detail requirements Community requirements – still valid? –EGI_DS project: http://knowledge.eu- egi.eu/knowledge/index.php/User_Requirementshttp://knowledge.eu- egi.eu/knowledge/index.php/User_Requirements –EGEE: https://savannah.cern.ch/support/?group=egeeptfhttps://savannah.cern.ch/support/?group=egeeptf

22. www.egi.eu EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 Discussion