1. 2nd SISAI meeting, June 2012
Policy for Setting up an Appropriate Environment for Storage, Distribution and Archiving of Statistical Software of Interest for the ESS
Doc 2.1.1
Marco Pellegrino, Krassimir Ivanov
Starting point:
ITDG 2011 requested a reflection on two issues:
Principles of IT Architecture (later in the agenda)
Developing a policy for setting up the appropriate environment for storage, distribution and archiving of statistical software of interest for the ESS (this ppt)
Unit B3, IT and standards for data and metadata exchange

2. Purpose Provide an overview of software developed
To establish a community allowing exchange and sharing of software (including OSS), economies of scale in using such software, applying harmonised rules
To foster a discussion on how to create an appropriate environment for storage, distribution and archiving of statistical software of interest for the ESS:
→ SISAI invited to comment
→ Discuss creation of task-force
First: several software tools are produced and maintained – either as OSS or based on proprietary solutions – and one of the major problems is having an overview of what is happening. Therefore, creating a framework in which common needs could be discussed, and the efforts shared, would benefit all participating parties.
In some cases, up-to-date information is accessible via Joinup or through other linked repositories (CIRCA, with summary documentation and hyperlinks on Joinup and on our SDMX Info Space).
Other kinds of software which are developed at national or international level are not equally visible, and no standard guideline is enforced on how to document their functionality, on support and on roadmap for further developments.
The objective of this document is to foster a discussion on how to create an appropriate environment for storage, distribution and archiving of statistical software of interest for the ESS.
The SISAI is invited to:
- provide comments and guidance on the issues highlighted in the document;
- discuss the proposal of creating a task-force that will further elaborate the coverage and content of the policy.

3. Software as a product with a life cycle (Software Development Lifecycle)
There are several models of the “software life cycle” describing the distinct phases of the software production process. Despite many variations, they all have very similar patterns and consist of a series of common stages which are repeated in a cycle, where each phase (analysis of user requirements, software design, development, testing and documentation, operation and maintenance) produces deliverables required by the following phase in the cycle.
User needs analysis, software design, development and testing
Software documentation and distribution
Deployment and maintenance, incl. support and training

4. Statistical Software Development
Continuous process carried out simultaneously at national level and at ESS level
National level: developments led by NSIs
ESS level: developments coordinated by Eurostat and based on strategic documents prescribing the required enhancements of statistical information systems and services
Software applications and systems are used at various steps of the ESS statistical production process complying with specific functional and technical requirements of these processes.
The work at national level is led by national statistical institutes, covering the needs of the community of users.
The analysis of software needs at ESS level is coordinated by Eurostat and is based on strategic discussions and documents prescribing the required enhancements of the statistical information systems and services.
The information about software needs at national level is channelled through the related task forces, working groups and committees where these topics are discussed by statistical domains or by "horizontal" areas of activities (e.g. ITDG, SISAI, GLC). These needs are reflected into development programmes at Eurostat in connection with financing and implementation decisions.

5. Work in progress
There are several software repositories allowing the sharing of software and related services within the ESS.
A number of software applications developed either by Eurostat or NSIs can be used and maintained.
To create favourable conditions for a wider sharing of software, the ESS should embark on a more consistent promotion of storage, maintenance and usage of software through shared repositories.
SDMX-RI and MA, registry, EDAMIS, DEMETRA, etc.
Modules which are evolving and are going to be more widely used

6. MSIS Software Inventory
Software Inventory Wiki of the MSIS Sharing Advisory Board (SAB)
Software sorted according to phases and sub-processes of the GSBPM or can be browsed through the alphabetical listing of software.
Eurostat software applications are included.

7. Software of interest for the ESS
Open Source Software (OSS) OSS for public administrations European Union Public Licence (EUPL) considered as a milestone SDMX-RI Mapping Assistant SDMX Registry
Commercial software
Software maintenance and user support Depending on the type of software (rules to be discussed)
EC promotes the use of OSS in public administrations through various actions.
Approval of the European Union Public Licence (EUPL) complemented by the foundation of various community platforms supporting software development and federating with other public administrations' similar platforms.
EC recently launched JOINUP which is a new collaborative platform offering services to help professionals share their experience, and supporting them to find, choose, re-use, develop, and implement software. Joinup also provides links to the work of other national and international repositories, encouraging a federation of software repositories and making projects more visible beyond the original community.
SDMX-RI and MA, registry, EDAMIS, DEMETRA, etc.
OSS is normally developed in a collaborative manner where programmers create a program and make it available for others to use - as well as modify - the source code and redistribute the modifications to the software user community. This is a challenging activity for a community, but it is normally based on the existence of a significant number of developers (which is not always the case for specific statistical software) and on a good visibility of results.
Our focus should be on software maintenance, evolution and support to users for which rules need to be discussed.

8. SDMX Tools Repository SDMX Tools Repository (on sdmx.org website):
SDMX web-site, “Tools” section:
“SDMX Tools Repository”:
Eurostat SDMX software:
Data Structure Wizard (DSW)
SDMX Converter
SDMX Registry
SDMX Reference Infrastructure (SDMX-RI)

9. Software repositories
created with specific objectives and operated to achieve specific aims
have their own storage environment or are only collections of links to other storage resources
Types of repositories:
Open Source Software (OSS) repositories
Proprietary software repositories
No certification of applications
Documentation supplied (variable level)
Several distinct repositories (others could be added) created with specific objectives and operated to achieve specific aims, having their own storage environment or links to other storage resources
Questions:
How to coordinate all this?
Is it possible to have a formal recognition, at the end of the development process, of the quality and applicability of the software, e.g. by giving a kind of "ESS certificate"?
Good standard documentation for users (different classes of users)
To guarantee the level of quality of OSS, the HLG-BAS (High-Level Group for Strategic Developments in Business Architecture in Statistics) has elaborated a document on "Principles and guidelines on building multilingual applications for official statistics" containing a set of specific criteria to be applied. The “owners” of the software will decide to what extend the software would meet the above guidelines.
A detailed specification of requirements for development of OSS for usage within the ESS ("Guidelines on publishing and sharing existing software as Open Source") is available to software developers on the Joinup website.

10. Conclusions Treat software as a product with a life cycle
Aim at a variety of users
Provide support (documentation, on-line help, training, guidelines)
Publish stable production versions, communicate roadmap
Create user groups?
ESS-certification?
To make collective use of software successful, treat software as product with a life cycle
Aim at a variety of users
Provide adequate support: no support is not an option
Publish a stable production version, communicate roadmaps
Facilitate self support at least (create user groups)

11. Statistical Software Policy task-force
Elaborates its mandate, method of work and scope of the policy
Establishes work plan and deliverables
Work mode mostly virtual (phone conference, wiki, physical meeting if required).
Draft to 3rd SISAI meeting (June 2013)
Approval at ITDG 2013
Lifetime: from September 2012 until November 2013