1. DDI 3.0 Conceptual Model Chris Nelson

2. Why Have a Model Non syntactic representation of the business domain Useful for identifying common constructs –Identification, versioning etc. –“patterns” A good basis for designing syntactic representation (e.g. XML) schemas, databases, and processing systems –Industry tools support this process (e.g. EMF)

3. Variable Scheme Physical Data Product Physical Instance Archive Study Unit Category Scheme Questions DDI 2.0 Driven by the need to archive data Developed as an XML DTD No formal conceptual model No re-use of artifacts

4. DDI 3.0 design goals – life-cycle model The statistical production process (Secondary) use of dataArchiving 2.0 3.0

5. Variable Scheme Physical Data Product Physical Instance Archive Study Unit Category Scheme Questions DDI 2.0

6. Identification, Item Schemes, Item Scheme Associations Component Schemes, Organisations Group Variable Scheme NCube Record Layout Physical Data Product Physical Instance Archive Study Unit Data/Metadata Resource Metadata Report DDI Base Structural Metadata Data/Metadata Management Category Scheme Concept Scheme Data & Metadata Structure Question Bank Instrument DDI 3.0

7. Identification, Item Schemes, Item Scheme Associations Component Schemes, Organisations Group Variable Scheme NCube Record Layout Physical Data Product Physical Instance Archive Study Unit Data/Metadata Resource Metadata Report DDI Base Structural Metadata Data/Metadata Management Category Scheme Concept Scheme Data & Metadata Structure Question Bank Instrument DDI 3.0 In Line NCube Record Layout NCube Table Layout NCube Layout

8. Instrumentation module A module in DDI 3.0 to describe survey instruments in a system independent way. and others To be used to drive data capturing systems or to pick up the output from these systems. Important metadata is entered at this stage and should be carried forward to the end data product. Information about question flow, cues presented to the respondents etc. is important for the interpretation of the data Often complex relationships between questions and variables. Q V V V

9. UML Constructs as used in the DDI Conceptual Model

10. Classes and Associations (1) 0..* 0..1 1..* 1 zero or more zero or one one or more one cardinalities

11. Classes and Associations (2)- Aggregates CategoryItem is subordinate to and “belongs to” CategoryScheme Aggregate by reference Aggregate by value In the model diagrams in this presentation there is no distinction made between aggregate by reference and aggregate by value. All aggregates are shown with a open diamond.

12. Classes and Associations (3)- Unidirectional Variable is navigable from DataAttribute but not vice-versa

13. Sub Classes - Inheritance DimensionVariable inherits from Variable (i.e. it is a “specialisation” of Variable). Therefore DimensionVariable can have an association to MetadataReport. However, any associations from DimensionVariable are specific to DimensionaVariable and are not applicable to Variable

14. Abstract Classes An abstract class is drawn because it is a useful way of grouping classes, and avoids drawing a complex diagram with lots of association lines, but where it is not foreseen that the class serves any other purpose (i.e. it is always implemented as one of its sub classes). Here Instrument inherits the attributes of Id, uri, urn. Instrument can have a multilingual name and description.

15. Instrument - Simplified Class Diagram

16. Question Bank - Simplified Class Diagram

17. Variable - Simplified Class Diagram

18. Identifying potentially comparative data The grouping mechanism can be used to mark up families of studies that from the outset have been designed to be comparable....or families of studies that has been made comparable through a harmonization process. However, none of these mechanisms reach beyond the limit of the DDI 3.0-wrapper that binds the family of studies together. One of the biggest challenges for DDI 3.0 has been to define a way to describe relationships between variables across DDI-wrappers, collections and servers. Use-case: “Give me more variables like this”, in other words the ability to identify potentially comparative variables across studies, collections, archives and locations.

19. Identifying potentially comparative data There is a mechanism in the existing DDI that to a certain degree will allow you do this. That is the ability to assign concepts from external vocabularies to variables. Study 1 V1V2V3 Study 2 V4V5V6 External vocabulary C1 C11C12C13

20. In DDI 3.0 there will a more elaborated solution to the same problem, a specification of an external registry-like question-bank or classification database that will allow you to register concepts, questions and variables. The specification can be used to set up local question banks or question banks that are global to many organizations. The specification will also support statements about differences between registered variables Study 1 V1V2V3 Study 2 V4V5V6 Identifying potentially comparative data External registry I1I2I3I4 Diff The registry can be seen as an extension to a standard DDI document....but the specification might also include the interfaces to allow this to be set up and run as a proper registry on the Web.

21. Registries Contains metadata that allows users/ applications to find things The objects themselves do not need to be in the registry –But must be accessible over the internet (preferably accessible by standardised queries and retrievable in a standardised format) –E.g. questions in question bank category schemes variables Registries can have repositories to store local content Registry standards exist and registry products are available –But they need to be customised to support the domain (e.g. customised software that understands the DDI model and syntax implementation) If objects can be identified in a globally unique way, then they can be accessed and shared

22. Data Analysis Data & Metadata Structure Physical Data Product NCube Record Layout

23. Cube Structure - Simplified

24. Data Structure NCube Record Layout NCube Logical Product Physical Instance

25. Cube Data – Contains or Points to Data Link to the Cube Structure Definition

26. DDI 3.0 Metadata Metadata constructs that are fairly generic and can be attached at various places in the hierarchy. Examples: –Coding instructions –Description of time and geography –Citation/Abstract –Methodology etc. The DDI model contains a metamodel for metadata structures: –Identifies the object types to which metadata can be attached –Specifies the category/concept schemes that contains the list of valid identifiers for the object types –Specifies the metadata reports that can be made (e. g. coding instructions, citation) in terms of Attributes Value domain (e.g. format) of the attributes Reporting hierarchy of the attribute –Identifies to which object types the metadata report can be associated

27. Metadata Attributes Object Identifier Metadata Structure Definition Identifier Components Item Scheme uses defined concepts defines the object types to which metadata can be “attached” specifies to which object types the report can be “attached” identifies the value domain of the component Metadata Report Concept Scheme concept defined in Concept takes semantic and context from Target Object Type identifies target object type of the component can have hierarchy Format and Permitted Value List Value domain identifies target object type of the identifier Specifies components for each Object (“key”

28. Metadata Structure

29. Metadata Set – Contains Metadata Reports

30. Modularity and grouping as a way to handle comparative data Ques- tions Study design Variab- les Group French study German study UK study Spanish study Italian study Extentions Local overrides Extentions Local overrides Translation Extentions Local overrides Translation Extentions Local overrides Translation Extentions Local overrides Translation

31. Modularity and grouping as a way to handle multiple tables/cubes Variables Study description nCube3nCube 2nCube1nCube5nCube4 Table description Table description Table description Table description Table description Group Category Schemes

32. Group: Logical Combination of Artifacts

33. Thank You