ABS Issues for DDI Futures Bryan Fitzpatrick October 2012

Use with Metadata Registry/Repository Store metadata once in MRR – probably as a Resource Package a “register/store” package, not a “use” package – use it by reference in all usage contexts We would like options for references everywhere – particularly in schemes Variable Schemes can reference Variables no other schemes can include items by reference

GSIM GSIM essentially aims to be the umbrella metadata context for Statistical Offices – based on the GSBPM – lots of things Statistical Offices want to help improve their processes Not really attempting to define “Actionable” metadata – happy to leave that mainly to DDI and SDMX – looking to them to “fill in” under GSIM

General GSIM Structures GSBPM-based – At lot of emphasis at the Specify Needs/Design stages – A lot of emphasis on the statistical structure Programs, Cycles, Activities Important “Context” metadata areas – in some ways analogous to DDI Groups and Sub-Groups Design away from “Silos” – explicit recognition of Data Resources separate Statistical Activities

General GSIM Structures Design away from Survey-centric focus – even-handed model for data acquisition survey, administrative, internet, other Process focus – DDI is very weak in this area – GSIM thinking is still evolving, incomplete Abstract Processes, Configured Processes, Executing Processes Base Processes on BPMN – but issues around designing for Process reuse – issues around linking to metadata – issues around configuring for statistical areas Functional metadata – DDI is good on instruments, weak elsewhere – GSIM will need metadata in all functional areas sampling, editing, imputation, derivation, weighting, confidentialising,... lots of thinking still to be done – probably mostly needs expression evaluation

General GSIM Structures Conceptual and Classifications – GSIM wants more sophisticated models for classification management critical area for Statistical Offices – management of evolution of large formal classifications » eg Industry, Occupation, Commodity – still need to for less formal, simple classifications Data Structures – attempt for model integration of unit-record and table structures DDI Unit Record model is good probably prefer SDMX model for tables maybe need to cope with more than file structures – Data integration

Context Metadata We see this as really important – I like the DDI Group/Sub-Group/Study Unit structure it encapsulates the context – We need a generalisation of that structure need to hold – statistical metadata of all types – process metadata and process configurations – “dataset” links – all mostly as MRR URNs – responsible units and people – “template” Study Units (generalised) – “Role-Forward” is big issues for statistical organisations Context metadata enables automation of this Also enables automation of the overall processes

Some nit-picking Annoying things in DDI – ordering of items in schemes can have contained items, items included from other schemes, perhaps included groups of items – but cannot order them! not important for some schemes (Concept, Variable) Critical for others (Code, perhaps Category) – Extensibility we always need options for including additional information in metadata DDI offering is Notes – not available in schemes and items » which is where you need tem we like SDMX Annotations – available everywhere – typed so conflicts do not arise

GSIM is important GSIM is the way Statistical Offices are planning to manage their business – if DDI want to be used in Statistical Offices underpinning GSIM is the obvious path – some tensions likely with data archive community but probably lots of commonality when you look closely (Some) Statistical Offices trying to manage the statistical process end-to-end – need “Actionable” metadata end-to-end ie DDI-type metadata