9 Feb 2004Mikko Mäkinen & Saija Ylönen Joint UNECE/Eurostat/OECD work session on statistical metadata (METIS) Geneva, 9-11 February 2004, Topic (ii): Metadata interchange / WP.6 MetaAPI – improving the usability of existing metadata systems by application programming interface

9 Feb Mikko Mäkinen & Saija Ylönen Statistics Finland’s metadata systems

9 Feb Mikko Mäkinen & Saija Ylönen Classification database (main menu screen) Classifications Correspondence tables Definitions Key words

9 Feb Mikko Mäkinen & Saija Ylönen Classification screen Code Label Metadata Metadata window

9 Feb Mikko Mäkinen & Saija Ylönen Classification versions in the database Classification versionText version

9 Feb Mikko Mäkinen & Saija Ylönen Problems with the present metadata systems User interfaces are somewhat outdated Technical and statistical metadata are intermingled There is no direct data connection to several file formats Text descriptions are not sufficiently structured and the description power of plain text is not adequate for mathematical formulas, figures, highlighting, etc. Training in application development does not yet have any practical connection to metadata systems

9 Feb Mikko Mäkinen & Saija Ylönen Production model project Improves the production processes in our office Covers collecting, storing, processing and disseminating of data Utilises modern technologies, tools and IT methods Data warehousing, software architectures, XML, web services, application development tools etc. Different kinds of metadata is needed in all phases of production We had no time (or knowledge) to find the perfect and final metadata solutions We had no time to rewrite everything we already had  A flexible, software independent solution with added change tolerance was needed to start with

9 Feb Mikko Mäkinen & Saija Ylönen MetaAPI structure MetaAPI = Metadata Application Programming Interface

9 Feb Mikko Mäkinen & Saija Ylönen MetaAPI services At the moment we have some basic classification services For getting codes, labels, definitions, key words Read only, internal only Basic services to produce large result sets concerning a single metadata object (e.g. one classification, statistical concept or survey data with all metadata concerning it) Schemas for the structure of objects are needed With these few services we can access all stored metadata Special services for more specialised common tasks (e.g. getting a label for a single code) Using “sub schemas” of the above

9 Feb Mikko Mäkinen & Saija Ylönen MetaAPI services in the future All classification services (including correspondence tables, etc.) Concept services Services for the metadata of data files Services to create and maintain metadata Services for mixed tasks (e.g. producing metadata rich publication tables) External services?

9 Feb Mikko Mäkinen & Saija Ylönen How does it work, what the user sees Example from our testing site: Getting classification codes and labels some server activity

9 Feb Mikko Mäkinen & Saija Ylönen How does it work, what really happens Getting classification codes and labels Data Access MetaServices EasyMeta TKMeta Classification API ConceptAPIUFileAPI Class. db Conc. db UFD db etc. SOAP request SOAP response...

9 Feb Mikko Mäkinen & Saija Ylönen MetaAPI implementation Server side.NET web service components (public interface).NET binary components (internal interfaces) Sybase and MS SQL Server databases to store metadata Client side.NET, PowerBuilder, SAS applications XML based systems in dissemination Browsers Anything (with support of using web services)

9 Feb Mikko Mäkinen & Saija Ylönen Change management We want to … have the possibility to adopt new standards and common tools in the future without changing everything … isolate the effects of system changes to a known area That is why … we should not allow direct connections to metadata systems and databases … all metadata business should take place through MetaAPI However, the uncertainty of the remaining lifetime of the present systems has slightly slowed down the speed of creating new services into MetaAPI

9 Feb Mikko Mäkinen & Saija Ylönen Change management Example: all background systems are replaced

9 Feb Mikko Mäkinen & Saija Ylönen Experiences so far MetaAPI has now been used in one real production system and in some test cases Experiences have been positive Web services are easy to use with modern application development tools XML skills are needed after receiving the result sets Selected techniques seems to be efficient (but no benchmarking has been done) Some client side optimisation may be necessary because XML documents can be large and parsing them can be time- consuming

9 Feb Mikko Mäkinen & Saija Ylönen What’s next? More services Using MetaAPI in new projects Guidance of users, client side tools, formatting layers, etc. Building connections to commercial software packages Data management decisions Adoption of common standards New user interfaces on top of MetaAPI, Metadata Portal External services