Ms. Marjukka Ala-Harja MoF, Finland, 1 The Quality Criteria of Public Online Services Mr. Juhani Korhonen Counsellor Public Management Department Ministry of Finance EPAN eGovernment working group 12 May 2005

Ms. Marjukka Ala-Harja MoF, Finland, 2 Quality development projects of public services since the beginning of 1990´s Experiences of other countries in developing the quality of public online services, EU-competitions The Action Plan of Public Online Services in Proposal to develop quality criteria for online services A co-operation project set by the MoF (September November 2004) Working methods: work of a project group, consultant, 3 brainstorming and testing groups (participants from ministries, public sector agencies, municipalities, civic organisations) Looking at the experiences of other countries (eg. The Netherlands, Denmark, Canada, UK) Background for the Work on the Quality Criteria

Ms. Marjukka Ala-Harja MoF, Finland, 3 The work done so far The Quality Criteria for public online services published in May 2004 The Quality Contest of public online services in Autumn 2004 A web-site has been set up ( supports the work on quality of online services presents the quality criteria and the results of the contest and provides links to other quality development tools

Ms. Marjukka Ala-Harja MoF, Finland, 4 The Purpose of the Quality Criteria 1) To serve as a tool for the construction, development and evaluation of public online services 2) To serve as the evaluation criteria of the Quality Contest of Public Online Services 2004 to find examples of high-quality online services 3) To improve the quality of online services especially from the point-of-view of the user, to increase customer satisfaction and thereby the use of online services.

Ms. Marjukka Ala-Harja MoF, Finland, 5 Developing the Criteria Organisational needs and benefits User needs and benefits Experiences of other countries Traditional quality assessment frameworks (eg. EFQM, CAF) Usability analyses of online services Quality Criteria Standards (eg. WAI), national recommendations and legislation

Ms. Marjukka Ala-Harja MoF, Finland, 6 The Challenge: Different Needs - One Set of Criteria Online service producers, technical planners, web designers, service providers, information officers, information specialists, managers, lawyers etc... Portals, information services, transaction services, simple and more complicated services… Usage by different experts Usage in different situations and phases Different online services Different evaluation perspectives Planning the online service for the first time, developing the existing online service, evaluating a service in a contest... Usability, accessibility, security, management, legislation, benefits, constructing the service, developing the processes Quality criteria as an intersection of various viewpoints and specialities

Ms. Marjukka Ala-Harja MoF, Finland, 7 Structure of the Evaluation Areas Benefits Use Content Management Production Evaluation of an online service from the viewpoint of the user by studying the online service Evaluation of processes behind an online service and an evaluation of the benefits obtained

Ms. Marjukka Ala-Harja MoF, Finland, 8 Altogether 40 Quality Criteria Use (14)Content (6) Management (4) Production(11) Benefits (5) Accessibility Ease of use User-friendliness of structure and layout Information content Interactiveness Link to organisational strategies & objectives Organisation of service production Evaluation of results Construction of the service User orientation Content production, maintenance, monitoring and development Ensuring security and functionality Benefits to user of the online service Benefits to organisation

Ms. Marjukka Ala-Harja MoF, Finland, 9 Example of a Criterion Evaluation area (5) Criterion (40) Characteristics of a criterion (2-7/criterion) Explanations, examples 1.Use 1.1 The public online service can be found easily The addresses of the public online service (the main address and other important addresses) are understandable and easy to guess. Long and difficult addresses are often derived from databases but they can also be easily simplified. The addresses of Finnish-speaking public online services are in Finnish. The fact that the most important addresses of the public online service remain the same in the new versions of the service also facilitates the guessing of the address.

Ms. Marjukka Ala-Harja MoF, Finland, 10

Ms. Marjukka Ala-Harja MoF, Finland, 11 Evaluation Tool (Excel) Evaluation area Characteristics and their grading Criterion Grading of the criterion Explanations, examples

Ms. Marjukka Ala-Harja MoF, Finland, 12

Ms. Marjukka Ala-Harja MoF, Finland, 13 Evaluation in a Group - A Learning Process for the Organisation Use Accessibility Easiness of use User-friendliness of structure and lay-out Information- security expert Director Content provider Evaluation areaEvaluation (examples of evaluators) Planner of the service Management Production Construction of the service User orientation Content provision, maintenance, monitoring and development Ensuring security and functionality Benefits Technical expert Content Information content Interactiveness

Ms. Marjukka Ala-Harja MoF, Finland, 14 The Quality Contest of Public Online Services in public online services as contestants national and local services of different sectors All the contestants completed a self evaluation using the evaluation tool 20 trained evaluators assessed the online services and also visited the organisations of the best contestants A five-person jury of influential persons from public and private sector The Prime Minister awarded the winners in November 2004

Ms. Marjukka Ala-Harja MoF, Finland, 15 Winners Portal Series: Education portal Opintoluotsi ( Individual Online Service Series: Local democracy service Avoin Espoo (“Open Espoo”) ( An honourable mention for the most innovative online service An application of the semantic web, museum portal MuseoSuomi ( The influence forum Vaikuttamo ( of youngsters of the Hämeenlinna surroundings.

Ms. Marjukka Ala-Harja MoF, Finland, 16

Ms. Marjukka Ala-Harja MoF, Finland, 17 Quality Contest Findings on Best Online Services The work done on improving the production and management of the service is clearly experienced by the end-user as a good and user friendly online service Integrating the users on constructing the service - it shows! A good online service does not emerge by unintentionally – a lot of work is needed Customer feedback is responded quickly and appropriately

Ms. Marjukka Ala-Harja MoF, Finland, 18 Quality Contest Findings: - Areas for further work Insufficient marketing of the services Excellent online services and contents, which have not been been properly found by users. Problems in marketing: money and attitudes. Work to be done on interactivity Most of the services still provide only information Benefits to be assessed Benefits of online services are identified but not measured or evaluated User satisfaction is being measured but not user benefits Even if the organisational benefits can be easily measured in euros that is not often done

Ms. Marjukka Ala-Harja MoF, Finland, 19 In the future Developing the website Laatuaverkkoon.fi: more detailed information of applicable legislation, information of best practices (eg. best alt-texts) etc. New contests within few years The national portal team ( has taken the responsibility of promoting the quality of public online services and developing the evaluation toolswww.suomi.fi

Ms. Marjukka Ala-Harja MoF, Finland, 20 Further information: (Quality Criteria also in English), Thank you for your attention!