1. Steps towards a Single Point of Access for Survey Questions across Europe: The Euro Question Bank Project
Wolfgang Zenk-Möltgen
Azadeh MahmoudHashemi
GESIS
FSD
FORS
UKDA
DDA
NSD
DANS
TARKI
SND
Consortium of European Social Science Data Archives
License: CC BY 4.0 (exceptions see last slide)

2. Agenda
Introduction
Purposes
Features
Use cases
Architecture

3. Introduction of Euro Question Bank
EQB content is provided by CESSDA Service Providers
Contains survey questions of different datasets in different languages
Contains associated information about studies, datasets, variables, etc
Expand on existing QDB for social science survey research
Offer databases of well-documented surveys and variables

4. Purposes of Euro Question Bank
Develop and implement a central search facility across all CESSDA surveys
Covering questions of surveys as much as possible
Exploration of findings on particular topics to identify existing survey items
Retrieval will be used for looking up question text, building new questionnaires or compare questions

5. Question Data bank Partners
TARKI
UKDA
NSD
DDA
New partners with animation
GESIS
SND
FORS
DANS
FSD

6. Key Features of CESSDA EQB
Based on DDI-Lifecycle metadata standard
Provide conversion tool from DDI-Codebook metadata standard
Search and filter by keywords, survey or series title, data collection dates or countries, question types or languages, provider, data availability, etc

7. Key Features of CESSDA EQB
Assist in searching in different languages by integrating multilingual thesaurus (ELSST)
Discovered questions include study-level, citation, frequencies, multilingual documentation and links to full original questionnaires
Access to CESSDA resources without switching systems

8. Basic Functions
A highly relevant tool for any kind of harmonization work
An important module in CESSDA Data portal
Present different versions of a question for immediate comparison
EQB will support the development of new questionnaire by giving researchers access to a selection of questions

9. Use Cases Import questions, variables, study from DDI-XML file
 Actor: Harvester
 Import through OSMH
Update questions, variables and study with import from DDI-XML file into DB, keep different version
 Actor: Provider
 Update automatic through harvester
Approach to build the use cases was to define the minimum viable product
This was a selection of the boarder list of functionalities
GESIS provide a possibility to import a specific type of DDI 3.2
No additional type of “older” metadata kept in EQB
Versioning  End users are only interested in fielded versions of questions.
The longitudinal study questions which have been modified over the waves, can be seen.

10. Use Cases Find questions, variables, study and concepts in EQB
Actor: End user
Filter and search facets on questions, variables, study and concepts in DB
 Actor: End user
3. EQB focuses on questions, not on variables. OSMH will have different repository handlers for DDL-C and DDI-L metadata. Therefore archives have flexibility of defining what constitutes a question. Some archives might represent sub-questions of complex questions as individual question included
Concept search should provide predictive term suggestions based on keywords actually used in metadata
4. Primary Obj of interest is Q. Study and variables can be treated as attributes, can be used for filtering and facet searches  Straight-forward technically.
EQB offers both simple and advanced search
 Attributes to use for facet searches (FA) or as filters (FI)
Study/group/series title (FA)
Question texts (targets ‘question text’, not pre-question or post-question text) (FA)
Response categories (FA)
Question language (Language of Research Instrument or File language??) (FI)
Fielded vs. translated questions (FI)
Country of collection (Nation) (FA or FI?)
Concept (optional) (FA)
Mode of collection  (relevant classes only) (FI)
Time method (relevant classes only) (FI)
Time frame (collDate at study level and/or Time period covered) (FA or FI?)

11. Use Cases
Compare documentation of questions, variables, study and concepts
Actor: End user
Explore relations between questions and question groups in a study
 Actor: End user
5. Select Qs and see them side by side. e.g. ICPSR & UKDA.
EQB will not provide any analysis of differences
In comparison page there should at least be the study title function as a direct link to detailed study description.
6. Select Q of interest to a basket to export questions, session must not end automatically

12. Use Cases Output for a new module, questionnaire or questions
Actor: End user
See how special metadata has developed and repeated in a survey
 Actor: End user
7. Display different versions (as defined by DDI 3.2 version of a question)

13. Use Cases
Translation for a new survey or a question related to a specific concept as well as switch the language to view other translation of the same question
 Actor: End user
Display related studies, datasets, variable names and lables which are used for specific questions
9. To be able to see different language versions of fielded questions, e.g. for doing new questionnaire translation, writing articles, designing a new module, comparing translations or language-country pair variants, etc.
10. Archives have information about which study/dataset a given question is connected to, and info about variables linked to a question.

14. Use Cases Usage statistics  Actor: Provider
Sort result list by preference
 Actor: End user
11. e.g. How much each filter has been used
12. Order the result list by preferences

15. EQB Architecture EQB – Frontend Vaadin UI EQB – Backend Elsatic Search
DDI-FlatDB
Open Source Metadata Harvester (OSMH)

16. EQB Architecture

17. EQB - Frontend User interface for faceted search
Similar to GESIS GLES question search
gles.gesis.org
Web services 
Vaadin as UI technology
Interacting between different web services
UX for user similar to desktop application
User interface for faceted search
Similar to GESIS GLES Question Search
Example see gles.gesis.org
Model and backend not recommended for reuse
Beginning of DDI-L model development with domain classes
Possible reuse
Our UI model for faceted query building
OR alternative framework support to query search index
Vaadin as UI technology
+ UX for user similar to desktop application
+ Component-oriented UI development possible
+ More stable code (avoid JS by developers)
- Statefull, object tree of user stored in his session

18. EQB - Frontend
Current state: Vaadin UI with simple search on study title implemented (accessing Elastic search index)

19. EQB - Frontend

20. EQB - Frontend

21. EQB - Frontend

22. EQB - Backend
Elastic search  Indexing  an open-source full-text search library
Search engine with JSON over HTTP web interface
Fast search responses using inverted index
DDI-FlatDB 
Idea is to access the entities very fast without any problem with DDI version or MySQL DB.
ES is able to achieve fast search responses, because instead of searching the text directly, it searches an index instead
This is like retrieving pages in a book related to a keyword by scanning the index at the back of the book, as opposed to searching every word
An index consists of one or more Documents, and a Document consists of one or more Fields
In DB, a Document corresponds to a table row, and a Field corresponds to table column
In ES an index may store documents of different “mapping types”. A mapping types is a way of separating the documents in an index not logical groups.

23. EQB - Backend DDI-FlatDB 
The DDI-Flat DB architecture is abstract, efficient, functional driven and REST-Full access to studies in DDI format.
Store Question, variable and study entities
Accessing and loading faster and easily
Current state: Elastic search index implemented (accessing Harvester), FlatDB implemented
DDI-FlatDB is flexible by all functionalities, models and configurations. The current DDI files are heterogeneous and varies over different versions.

24. Open Source Metadata Harvester (OSMH)
Harvest all information from heterogeneous and autonomous handlers (SPs) with different technologies
A CESSDA MH classify the entities and objects which gets harvested
Repository Handlers Enables repository owners to write RHs for repository technology they use
Current state: Repository Handlers for NESSTAR servers implemented
Partners within the project extend the existing OSMH by additional metadata fields to support our use cases

25. Questions?

26. Thank you for your attention

27. License This presentation is offered under license CC-BY 4.0.
The license does not apply to the following copyrighted material used in this presentation:
The logo of GESIS
The logo of CESSDA
The slideshow layout of CESSDA