Intro The Evaluation of the NEFIS Project focused on: Metadata Data Rights and Data Rights Management User Needs and User Expectations - VTK

Evaluation based on two approaches: Intro Evaluation based on two approaches: Standardised questionnaires metadata and metadata guidelines data rights and data rights management user needs and user expectations - VTK Expert Statements metadata and metadata guidelines Prof. Dr. Keith Rennolls (Greenwich), Dr. Jarmo Saarikko (Metla) keyword list and thesaurus Renate Prüller (IUFRO), Gillian Petrokofsky (CAB International) data rights and data rights management Andreas Schuck (EFI), Tim Green (EFI) EFIS – an electronic system for periodic reporting Prof. Dr. Michael Köhl (UHH), Aljoscha Requardt (UHH)

Metadata Evaluation Metadata 1. Elaboration of metadata records a step-wise iterative process - preliminary records according previous schema - metadata schema developed further to its final version (D3, D5) - revise preliminary records make adjustments according new schema 2. Questionnaire “Metadata and Metadata Guidelines” - evaluate experienced applicability and usability 3. Questionnaire results represented and discussed at the Hamburg Meeting Expert Statements - metadata and metadata guidelines - keyword list and thesaurus Evaluation Metadata

Metadata Questionnaire “Metadata and Metadata Guidelines” Knowledge and experiences of the respondent Workload Metadata Guidelines NEFIS Metadata Schema Specific Metadata Elements and Refinements Metadata Functionality and Structure Future Perspectives

Metadata General Are the NEFIS metadata the first contact you have had with metadata? 79% Yes Partners knowledge and experience of elaborating metadata Partners knowledge and experience using metadata for data source retrieval

Time Investment - most partners between 5-10 hours in total Metadata Workload Do you consider the workload to enter metadata records as: Level of complexity to prepare and enter metadata: 71% = acceptable 29% = complex Time Investment - most partners between 5-10 hours in total

Metadata Guidelines Are the guidelines easy to understand? Would a metadata tutorial or training course be helpful and appropriate? 71% Yes

Metadata Schema Metadata records without problems

Metadata Schema Workload: time to prepare and enter required information

Metadata Schema Relevance - data and resource documentation

Metadata Schema Relevance - data retrieval

Metadata Specific Issues Type Format Coverage Quality Report Subject DatasetGeoreferenced Format Reference System - encoding scheme for georeferenced data Coverage Point / Box encoding scheme Quality Report Subject Audience

Metadata Quality Report "Quality" addressed within DCMI elements, refinements and encoding schemes? Named elements which address “Quality” within the DCMI schema: Creator, Description, Publisher, Coverage, Source, Date, Audience

Metadata Quality Report Relative value of a “Quality Report“ “Quality Report“ under the element “Descriptions“?

Metadata Quality Report Value of listed options to describe and structure “Quality Report"

Metadata Quality Report Value of quantitative measures of quality (standard error, sample size, sampling unit, resampling for measurement control) Which of these quantitative measures are relevant and appropriate for which type of data? it could become very complicated and time consuming to collect such information in a standardised way.

Metadata Subject Can the dataset be appropriately described by using the NEFIS terms? sometimes difficult to group the term under a theme very time consuming to go through the lists … more structure/ hierarchy would be helpful.

Metadata Subject How important is the option to add "Nominated Terms" it is the way to improve the term lists very important during the development phase …it allows the provider to demonstrate which additional terms are seen necessary to describe the particular resource more accurately

Applicability and Usability Metadata Schema - Resume Metadata Element 1. Title 2. Creator 3. Subject 4. Description 5. Publisher 6. Contributor 7. Date 8. Type 9. Format 10. Identifier 11. Source 12. Language 13. Relation 14. Coverage 15. Rights 16. Audience Applicability and Usability = easy ?? = difficult (hot spots) ? = acceptable ? ?? ?? ? ?? ? ? ? ? ?

Metadata Conclusion Obstacles within the challenging tasks as the development of keyword lists as well as the quality report could have been clearly reflected by the results of the metadata evaluation Subject, Description and Format caused problems for most partners low experience by some partners complicated structure and defined procedures NEFIS metadata schema is applicable in its current form from the perspective as a data provider – dataset documentation from the perspective as a data user – data resource retrieval

Data Rights Data provider has to state what data are accessible and for whom, and which type of data rights and which levels of access should be assigned to which kind of data. Data provider should have options to assign different levels of access to individual datasets in order to protect sensitive data by restricting access to them. Prerequisite to guarantee that legal restrictions to data access will be satisfied

Data Rights Why to talk about Data Rights? Data rights provoked several discussions “Data Rights”, does it concern the actual data? Should not all data presented in NEFIS be “public”? NEFIS tables are available for free. On the other hand, same information is part of an on-line system to which you have a restricted access. How should we handle this? Clarify Data Rights within the NEFIS project beyond the time of NEFIS project

Data Rights Which of the "drop down" options (if any) are relevant to your data set? Are these “drop down“ options appropriate for your data set? 95% Yes But option “Public” needs further refinements

Data Rights Who is responsible for data access management? Which form of data storage do you prefer for your data set?

Data Rights Would you like to provide more data? Are there any obstacles to provide additional data? cost for the provider ! cost for the user technical limitations unclear data rights management

Data Rights Partners demands for further cooperation and data provision Technical demands and demands on data management link data easily to the system without complicating and restricting data management and maintenance clear definitions and standards for data format and data content Financial demands capacities for time and personnel investments

User Needs and User Expectations - VTK 1. VTK Workshop 2. anonymous comments by partners regarding VTK display and analysis options 3. Test CD 4. Questionnaire “User Needs and User Expectations” 5. Expert Statement (UHH) “EFIS - an electronic system for periodic reporting” Evaluation „VTK & Information needs“

User Needs and User Expectations - VTK Questionnaire “User Needs and User Expectations” The respondet Usability Performances VTK features Tutorial and Help Functions Improvements Future Potential

User Needs and User Expectations - VTK The respondet A - familiar with the NEFIS project and the provided dataset B - only familiar with the provided dataset itself C - someone who is not familiar with the NEFIS project and the provided dataset, but has some broad knowledge in using GIS applications and statistical programs Questionnaire evaluated according: Total GIS Knowledge Statistical knowledge 19 answered questionnaires Expirences of useing the VTK = average / less than average Respondent spent 7 hours using the VTK (5 hours for one explicit dataset)

User Needs and User Expectations - VTK How easy is it to find applicable and reasonable functions to analyse and visualise data which are of explicit interest? functions are atractive but it is not easy to find out how to use them and select the one that is needed VTK examples are an important addition to get a more transparant view which of the features are useful for which kind of analysis

User Needs and User Expectations - VTK Respondent meaning about the VTK menu the menu should be developed more simple and informative some better “getting started directions” would be very helpful (essential even).

User Needs and User Expectations - VTK The VTK incorporates automated map design the produced maps are interactive, including traditional GIS interactivity and visual manipulations outputs are available as data tables and thematic maps with different techniques for selected geographical area it allows production of statistical graphics (box plots, dot plots, scatter plots, etc.) that are dynamically linked to maps This could imply that expert knowledge on cartography or GIS is not required not necessariliy knowledge in GIS but some knowledge in cartography and data presentation the user should be familiar with the data itself

User Needs and User Expectations - VTK Consideration of options to analyse available datasets Consideration of options to visualise available datasets

User Needs and User Expectations - VTK Ability to recall explicit information from displayed formats Mapped data Data plots Tabular data high Figures satisfactory Units unsatisfactory Lack in the actual dataset itself !!!

User Needs and User Expectations - VTK VTK offers many features, tools and options to analyse and to visualise data the feature should be applicable to the dataset the user should understand meaning and relevance of the feature the user should be able to recall relevant information after using a suitable feature the appearance of the outputs should be satisfactory for the user no essential functionality in that feature should be missing

User Needs and User Expectations - VTK

User Needs and User Expectations - VTK Most respondents considered all listed applications as important or as interesting but the respondent was not always able to test some explicit applications, mainly due to the lack of help functions and tutorial documents further tutorial documents and help functions can be seen as a useful and important asset !

User Needs and User Expectations - VTK Time investment to understand and use VTK features high Complexitiy to understand available VTK features high Suggestion: Develop a set of tutorial tailored at different user groups and level of expertise

User Needs and User Expectations - VTK Reduce the set of VTK features or Add new features ??? Suggestion: Develop a VTK for two different groups of users Basic Version – new users Advanced Version – experienced users Key: User Friendliness - improved interfaces - improved maneuvering through features - improved guidance for getting started Incorporating too many purposes into one system component will logically lead to the consequence of not being able to meet all these demands. Therefore it is reasonable to differentiate the use but also the applications of the system. Reporting reporting schemes reporting query options

User Needs and User Expectations - VTK VTK functional and applicable for your purposes as a data provider?

User Needs and User Expectations - VTK User Backgrounds and Expectations are different Simplified self explanatory output features Clearly structured user guidelines Easy to follow help functions Define the purposes of the system – Who are the potential users? Differentiate the use and applications Reporting most important purpose - guarantee that VTK outputs not ambiguous in their statement - contact data user – data provider Although some applications could be developed further or improved, the VTK and its features provide a solid basis to meet various tasks of a full working EFIS.

Thank you