The Library of Online Harmonisations Martin Friedrichs and Kristi Winters EDDI 2016 License CC BY 4.0

Introduction Harmonizing variables is an important part of social science research. But for variable harmonization work to be of scientific value the harmonisation process must be documented and published in a precise and transparent ways. However even if done so these intellectual contributions are often lost because researchers can’t publish their work. Our solution is an online library where digitally documented harmonisation routines will be archived and accessed.

Overview L.o.H

“Nice. But what is the relationship to DDI?” Question “Nice. But what is the relationship to DDI?”

Answer Such a platform requires metadata and data harmonisation workflow to be represented in a standardized and machine actionable way. As it is now harmonisation routines arise from different sources, like SPSS, Stata or as output from applications like CharmStats. These very different types of harmonisation routines should be represented by a single standard inside the library. We also need a standard for exchanging said “digitally documented harmonisation routines” with tools outside the library. Our idea is to combine these two requirements and to use an XML based representation for both of them. Which brings us to DDI.

DDI

Larry Hoyle and Joachim Wackerow „DDI as a Common Format for Export and Import for Statistical Packages“

In this article published 2015 in IASSIST Quarterly the authors presented an experiment using Stat/Transfer to exchange datasets between software packages with DDI Lifecycle as the „standard representation“.

Their conclusion „Adoption of DDI by tools like Stats/Transfer is encouraging. Basic metadata is transferable among all 5 packages via DDI. The current state still means that some important metadata […] still must be either hand entered into DDI or harvested and entered by user-witten code.“ (pg. 20)

Digitally documented harmonisation routines Charmstats

About CharmStats (CS)

Overview CS Datamodel Study AttributeComp Question Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

Overview CS Datamodel Study AttributeComp Question Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

Overview CS Datamodel Study AttributeComp Question Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

Overview CS Datamodel Study AttributeComp Question Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

Overview CS Datamodel Study AttributeComp Question Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

Overview CS Datamodel Study AttributeComp Question Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

The Hidden Elements Measurement Dimension Indicator Variable

Overview CS Datamodel InstanceLink S. Inst. AttributeLink Measure CharLink Category InstanceLink C. Inst. AttributeLink Dimension CharLink Specification InstanceLink O. Inst. AttributeLink Indicator CharLink Prescription Project InstanceLink D. Inst. AttributeLink Variable CharLink Value

Overview CS Datamodel InstanceLink S. Inst. AttributeLink Measure CharLink Category InstanceLink C. Inst. AttributeLink Dimension CharLink Specification InstanceLink O. Inst. AttributeLink Indicator CharLink Prescription Project InstanceLink D. Inst. AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

Overview CS Datamodel InstanceLink S. Inst. AttributeLink Measure CharLink Category InstanceLink C. Inst. AttributeLink Dimension CharLink Specification InstanceLink O. Inst. AttributeLink Indicator CharLink Prescription Project InstanceLink D. Inst. AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

Overview CS Datamodel InstanceLink S. Inst. AttributeLink Measure CharLink Category InstanceLink C. Inst. AttributeLink Dimension CharLink Specification InstanceLink O. Inst. AttributeLink Indicator CharLink Prescription Project InstanceLink D. Inst. AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

DDI and the CS Model

Variable Variable VariableType Name VariableName Label r:Label Level r:VersionableType Variable VariableType Name VariableName Label r:Label Level MeasureType Question r:QuestionReference Dataset DatasetLabel PID DateLastChanged r:Description … Temporal* isTemporal Geographic* isGeographic isWeight

Variable Variable Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

Instance Instance VariableSchemeType VariableGroupType Label r:Maintainable r:Versionable Instance VariableSchemeType VariableGroupType Label VariableSchemeName VariableGroupName r:Label Type TypeOfVariableGroup PublishedSince* ObsoleteSince* Description r:Description r:VariableSchemeReference r:UniverseReference r:ConceptReference r:Subject r:Keyword Variable r:VariableReference VariableGroup r:VariableGroupReference isOrdered

Instance V. Scheme V. Group Variable Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap

References, Links and Maps InstanceLink AttributeLink CharacteristicLink ID Project Instance Attribute Characteristic InstanceLinkType AttributeLinkType CharacteristicLinkType InstanceMap AttributeMap CharacteristicMap ID SourceInstanceLink SourceAttributeLink SourceCharacteristicLink TargetInstanceLink TargetAttributeLink TargetCharacteristicLink InstanceMapType AttributeMapType CharacteristicMapType Project ConceptRef Referenced Concept RefRelationshipType <xs:complexType name="ReferenceType"> <xs:sequence> <xs:choice minOccurs="1" maxOccurs="2"> <xs:element ref="URN"></xs:element> <xs:element ref="Agency"></xs:element> <xs:element ref="ID"></xs:element> <xs:element ref="Version"></xs:element> </xs:sequence> </xs:choice> <xs:element ref="TypeOfObject"></xs:element> <xs:element ref="MaintainableObject" minOccurs="0"></xs:element> <xs:attribute name="isExternal" type="xs:boolean" default="false"></xs:attribute> <xs:attribute name="externalReferenceDefaultURI" type="xs:anyURI" use="optional"></xs:attribute> <xs:attribute name="isReference" type="xs:boolean" fixed="true"></xs:attribute> <xs:attribute name="lateBound" type="xs:boolean" default="false"></xs:attribute> <xs:attribute name="lateBoundRestriction" type="VersionType" use="optional"></xs:attribute> <xs:attribute name="objectLanguage" type="LanguageList" use="optional"></xs:attribute> <xs:attribute name="sourceContext" type="xs:anyURI" use="optional"></xs:attribute> </xs:complexType>

AttributeLink? Problem: We can‘t refer to r:VariableRef V. Scheme V. Group r:VariableRef Variable Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap Problem: We can‘t refer to r:VariableRef

„AttributeLink“ Variable V. Scheme V. Group Project InstanceLink CharLink Value InstanceMap AttributeMap CharMap

AttributeMap Variable V. Scheme V. Group Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap V. Scheme V. Group r:VariableRef

Project ResourcePackage Variable V. Scheme V. Group Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap V. Scheme V. Group r:VariableRef

„InstanceLink“ ResourcePackage Variable V. Scheme V. Scheme V. Group Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap V. Scheme V. Group r:VariableRef

„InstanceMap“ ResourcePackage Variable V. Scheme V. Scheme V. SchemeRef V. Group Project InstanceLink Instance AttributeLink Variable CharLink Value InstanceMap AttributeMap CharMap V. Scheme V. Group r:VariableRef

Conclusion To replicate the CS model as it was implemented with DDI is not possible The simulate the idea behind the CS model with DDI might be possible Further research is necessary

Thank you for your patience!