Lessons Learned, Future Plans and Conclusions ETICS2 Periodic Review

Lessons Learned ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

The ‘Four Major Challenges’ Effort-constrained projects Even projects that do believe that doing testing and QA gives benefits, often do not have enough effort to write tests. They like ETICS, but when they realize that ETICS doesn’t write the tests for them, they adopt the build part, but postpone the adoption of the test part to ‘more relaxed times’ Our insistence on showing the benefit of automating the test processes is slowly paying off. Most of the new registered or potential projects (UNICORE, ARC, StoRM, Edges) actually are more attracted by the test capabilities and in particular by the multi-node features. The possibility of doing complex interoperability tests is appealing to many test engineers. ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

The ‘Four Major Challenges’ Developer-centric decision processes In many occasions technical decisions are taken bottom-up, but every developer has different habits or preferred tools and tends to resist to changes especially when he/she is afraid of ‘loosing control’ of the software ETICS was initially targeted to integrators, testers and release managers. This may be effective in process-driven environments, but it’s not completely realistic in the academic software development world. We also have to acknowledge than 90% of the ETICS users are developers. We have now added or are adding changes in the system that appeal more to developers (better performance, wizards for common tasks, configuration import/export tools, integration with lower level build tools, etc). This has lowered the barrier to entry for new developers and helped in improving the expectations. ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

The ‘Four Major Challenges’ Software not designed for automation Some software we have dealt with in the past two years is not designed for test automation. The deployment and configuration operations are often manual and do not provide ways of easily scripting or instrumenting the procedures Although this remains true for those project, it is not the rule. Both existing and new users like D4Science, UNICORE and ARC have shown that the software can be automatically deployed and tested. We are also helping in improving the situation when automatic deployment is difficult by providing ready-made scripts and templates for deployment tests. An important role is played by cross-disseminating of the achievements of more advanced projects as part of our demos and tutorials (example D4Science deployment test demo for gLite testers). ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

The ‘Four Major Challenges’ QA-unaware projects A number of short-term projects are mainly concerned with releasing as soon as possible and believe that doing testing slows their work and cannot improve the quality within the projects lifespan This is slowly changing because many of the projects we are working with are actually continuations of older projects and middleware and applications are gradually consolidating. We have observed a greater attention to quality. Additionally the interest for validation of standard compliance and interoperability is growing stronger and we have noticed strong interest for the possibility of providing a repository of shared, ready-to-be-used testsuites and benchmarks. For example this has been one of the outcomes of the ‘Grid research’ session at the recent OGF 25 conference. ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

New Requirements Easier resource access, security, privacy Many small or commercial projects find it difficult to use ‘grid’ standards They are not used to use client certificates and prefer passwords Commercial projects or companies cannot use the standard Geant backbone and need cost-effective resource provision solutions to implement large distributed testing We have received questions about the possibility of using ‘clouds’ with ETICS The lack of information privacy in ETICS is considered by commercial users a showstopper Although this was already known, the commercial viability requirement analysis work done IN SA2 (especially by VEGA) has shown that the priority of this feature must be raised ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

Future Plans ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

Infrastructures, Services, Technologies Release the new submitters in production and start offering pilot build/test services within EGEE Investigate the possibility of service deployment on UNICORE and ARC-based infrastructures Implement and deploy on-demand virtualization Implement data confidentiality across the system Add import/export converters for popular tools or services like Maven, CMT, SourceForge, etc (depends on user feedback) Finalize the new registration and test workflow design tools Expand the portfolio of new projects with special focus on commercial applications ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

Certification, standardization and dissemination Finalize and deploy the A-QCM Report Generator Activate the certification service starting the first trial certifications with ETICS, gLite and D4Science Start the standardization process of A-QCM Finalize the sustainability strategy and define the transition plan Keep the high level of dissemination and training ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

Conclusions ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

Summary ETICS is playing an important role in improving the awareness about the benefits of software quality and standardization It has become a platform where different projects can share expertise and investigate interoperability and compliance issues We have identified requirements that were not known at the beginning and may improve the suitability of ETICS beyond the research environments However, we need to prioritize. Making ETICS more sustainable (data confidentiality, easier resource access) may require being more selective in our commitment to existing projects The objectives for the second year are clear and all project members are committed to achieve them ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009

http://www.eticsproject.eu Thanks ETICS 2 Periodic Review - Lessons Learned - Brussels, 3 April 2009