1. CountryData SDMX for Development Indicators
MDG DSD vs. the Di Database:
Using the Mapping Tool

2. MDG Data Structure Definition (DSD) background
Developed by SDMX Task Team of IAEG on MDGs
Supports exchange of MDG Indicator data between international agencies (UN, UNICEF, UNESCO, …)
Implemented in SDMX 2.0
Latest version (2.4) finalised in Feb 2013

3. DevInfo (Di) background
Data dissemination software supported
and promoted by UNICEF
DevInfo7 (Di7) launched in Nov 2012
SDMX 2.1 & 2.0 compliant
Web base software
9 out of 11 project countries using DevInfo
Stable version compare to previous releases

4. Simple relation between Di & DSD
Di Database
MDG DSD
Frequency (Default = “Annual”)
Reference Area
Series
Units of measurement
Unit multiplier (Default = 0)
Location (Default = “Total”)
Age group (Default = “All Ages”)
Sex (Default = “Both Sexes”)
Source Type (Default = “NA”)
Source details
Time Period
Time period details
Nature of data points (Default = “C”)
Footnotes
Area
Indicator
Unit
Subgroup (i.e. Sex, Age, Location etc.)
Source
Time Period
Footnotes

5. Mapping to the DSD DSD dimensional structure means
values are mandatory for LOCATION, SEX & AGE GROUP.
Due the nature of this domain (i.e. MDGs), not obvious which values should be used in these dimensions
For example, what is SEX for “Births attended by skilled personnel”:
Not Applicable? Total? Female?

6. Mapping to the DSD
Inconsistent mappings lead to duplications and other anomalies
In CountryData, mappings for indicators/ time series are agreed before data exchange (see mapping for MDGs from 1st workshop)
However, this is just one side of the story…

7. Mapping to the DSD
Understanding the structure and contents of the origin database is fundamental to the mapping process
Mapping to the DSD requires the data to enter into certain ‘restrictions’ it’s not bound by in the database (and vice versa).

8. Mapping to the DSD The mapping tool in di software is
designed to work with the di database as simply as possible…
the tool is based on mapping between the codelists of the DSD and origin database;
certain situations require some further manual effort to map a time series;
and sometimes a “fix” is required to the database where the data simply isn’t valid or it’s duplicated.
Therefore it’s good to review di structure to understand where these issues usually occur.

9. Di Data Architecture Area, hierarchical dimension
IUS = Indicator, Unit and Subgroup
Time series data are stored with the combination of the 3 dimensions
Indicator
Unit
Subgroup: Combination of one or more sub-dimensions
Source & Time Period
Together with IUS “uniquely” defines each data value
Footnote
“Free text” field stored with data value

10. Di INDICATOR IUS: Indicator Unit Subgroup Indicator, for example:
Infant Mortality
AIDS Death
Malaria Death
Similar to SERIES in the DSD
Contains only Indicator specific values

11. Di UNIT IUS: Indicator Unit Subgroup Unit:
Percentage
Number
USD
Square KM
Similar to UNIT of Measurement in DSD
Contains only Unit specific values

12. Di SUBGROUP Combination of one or more sub-dimensions
IUS: Indicator Unit Subgroup
SubGroup Dimension:
Combination of one or more sub-dimensions
“Age”, “Sex”, “Location” and “Other” sub-dimensions are set initially in database
Specific values can be created under each sub-dimension
Relate to SEX, AGE GROUP and LOCATION in DSD.

13. Di SUBGROUP IUS: Indicator Unit Subgroup Formation Logic:
Sub-Dimension
Age
Sex
Location
Other
Sub-Dimension values
< 1 Year
< 5 Year
5 – 10 Year
Male
Female
Urban
Rural
Total
Rice
Wheat
SUBGROUP
(Combination)
<1 Year Male
<5 Year Female Rural
Urban

14. Di SUBGROUP

15. Di SUBGROUP

16. Di SOURCE

17. Di TIME PERIOD

18. Di Mapping Tool: Introduction
Once data exists in di7 web-based software then data can be mapped and published which conforms with the MDG DSD.
This is all done online through the di7 web-based repository through the administration profile, so let’s begin…

19. Getting Started…

20. Scroll down to ‘Registry’ menu

21. Log onto administrative profile

22. Log onto administrative profile

23. Full access to ‘Registry’ features

24. Prepare the Dbase for mapping

25. Prepares the SDMX artefactes

26. Ready to ‘Upload’ the DSD

27. Choose a DSD from your folders

28. Choose a DSD from your folders

29. DSD Upload is a success…

30. Now you are ready to map…

31. 1st Step: Codelist mapping

32. 1st Step: DSD Codelists Indicator CodeList SH_HIV_INCD
HIV incidence rate
SH_MLR_MORT
Notified cases of malaria
SE_ADT_1524
Literacy rate
SE_PRM_CMPL
Primary completion rate
Location CodeList T
T Total (national level) U
U Urban R
R Rural
SEX CodeList NA
Not applicable F
Female M
Male T
Both sexes
UNIT CodeList NA
Not applicable
CUR_LCU
Local currency
USD
NUMBER
Number
RATIO
Ratio
PERCENT
Percent
KM2
Square kilometers T
Metric Tons
PER_100_LIVE_BIRTHS
Per 100 live births
PER_100_POP
Per 100 population
PER_1000_LIVE_BIRTHS
Per 1,000 live births
PER_1000_POP
Per 1,000 population
PER_100000_LIVE_BIRTHS
Per 100,000 live births
PER_100000_POP
Per 100,000 population
AGE CodeList  NA
Not applicable
000_099_Y
All age ranges
000_006_M
under 6 month olds
000_005_Y
under 5 year olds
000_001_Y
under 1 year olds
000_018_Y
under 18 year olds
000_006_Y
under 6 year olds
010_005_Y
10-14 year olds
015_005_Y
15-19 year olds
015_010_Y
15-24 year olds
015_035_Y
15-49 year olds
006_054_M
6-59 months old
006_009_Y
6-14 year olds
005_013_Y
5-17 year olds
015_050_Y
15-64 year olds

33. 1st Step: (A) Map Indicator codes

34. 1st Step: (B) Map Unit codes

35. 1st Step: (C) Map Subgroup codes

36. 1st Step: (C) Choose Subgroup list

37. 1st Step: (C) Map Age subgroup

38. 1st Step: (C) Map Sex & Location

39. 1st Step: (D) Map Area

40. 1st Step: Save codelist mappings

41. 1st Step: Ignore warning

42. 1st Step: Confirm mapping saved

43. Exercise 1: Codelist mapping
Use unstats.un.org/unsd/demodiweb[1-6]
Username =
Password =
Map the codelists (where possible) for
Unit
Age
Sex
Location
Area
And just one indicator, “Antenatal care coverage for at least one visit”

44. 1st Step: Complete

45. 2nd Step: Confirm IUS mapping

46. 2nd Step: Save IUS Mappings

47. Exercise 2: mapping time series
Use unstats.un.org/unsd/demodiweb[1-6]
Username =
Password =
Map the time series for
“Antenatal care coverage for at least four visits”
“Employment to population ratio”
“Literacy rate of year-olds”
“Death rate associated with malaria”
“Proportion of population using solid fuels”

48. 2nd Step: Complete

49. Final Step: Register the mappings

50. Final Step: Select mappings

51. Final Step: Generate SDMX-ML

52. Final Step: Complete

53. Exercise 3: Publish time series
Use unstats.un.org/unsd/demodiweb[1-6]
Username =
Password =
Publish/ register the time series for
“Antenatal care coverage for at least four visits”
“Employment to population ratio”
“Literacy rate of year-olds”
“Death rate associated with malaria”
“Proportion of population using solid fuels”

54. Why the 2nd step?
The default values for SEX, LOCATION or AGE GROUP mapping may not be applicable to all mappings
The codelist mapping may only provide a partial mapping of the time series (i.e. more information is required)
These changes are made in the 2nd step.
This is all done online through the di7 web-based repository through the administration profile, so let’s begin…

55. Where are the default values?

56. Admin panel: Application settings
Insert screens shot/details of admin panel and default value storage…

57. Application settings has all mapping default values

58. Manual mapping of SUBGROUP
Where a subgroup value is missing the default values will apply, for example…
Unit
Indicator
Default Values
Location = T … ?

59. Manual mapping of SUBGROUP
So subgroups coverage affects the number of manual changes which have to be made…
Indicator
Unit
Subgroup for Age and Sex?
Default Values …
Age Group = 000_099_Y
Sex = T

60. Manual mapping of SUBGROUP
Common example of where default subgroup mapping do not apply
Indicator
Unit
Subgroups?
Default Values
Location = T
Age Group = 000_099_Y
Sex = Both sexes

61. Manual mapping of SUBGROUP
Common example of where default subgroup mapping do not apply
Subgroup for Sex?
Indicator
Unit
Default Values …
Sex = T

62. Manual mapping of SUBGROUP
Common example of where default subgroup mapping do not apply
Indicator
Unit ?
Default Values
Location = T
Age Group = 000_099_Y
Sex = T
Subgroup for Location, Age and Sex?

63. Manual mapping of SUBGROUP
If the subgroups are sorted more simply, this also helps with the mapping:
Indicator
Unit
Default Values
Location = T … ?

64. Manual mapping of SUBGROUP
Common example of where default subgroup mapping do not apply
Indicator
Unit ?
Default Values
Location = T
Age Group = 000_099_Y
Sex = T
Subgroup for Location, Age and Sex?

65. Back to mapping…

66. 2nd Step: Amend Indicator
When using the check box to tick the mapping, you are “fixing” the mapped DSD values. If the box is unchecked again and the mappings saved, then DSD values revert to those mapped at codelist/ default values (i.e. any manual changes are undone.)

67. Final Step: Register new mappings

68. Exercise 4: Amend time series
Use unstats.un.org/unsd/demodiweb[1-6]
Map/ amend/ publish the time series for;
“Antenatal coverage rate”
“Children orphaned by AIDS”
“Children under-five sleeping under insecticide-treated net (ITN)”
“Proportion of births attended by skilled health personnel”
“Share of women in wage employment in the non-agricultural sector”
“Proportion of urban population living in slums”

69. More complex mappings under the 1st and 2nd mapping step?
The most common changes made to mappings are between subgroups and the Sex, Age Group and Location dimensions
But sometimes manual changes are required between di and DSD indicator and unit, either…
More than one di code relates to a single DSD code OR
More than one DSD code relates to a single di code

70. Many-to-one mapping for Indicator codelist (Example 1)

71. Many-to-one mapping for Indicator codelist (Example 2)

72. Many-to-one mapping for Indicator codelist (Example 3)

73. Many-to-one mapping for Unit codelist (Example 1)

74. Manual mapping of INDICATOR?
Unit
Manual change

75. Manual mapping of INDICATOR
Unit
Indicator ?
Manual change

76. Manual mapping of UNIT
Indicator
Unit
Manual change
Unit = “Ratio”

77. Back to mapping…

78. 1st Step: many di to 1 DSD code

79. 2nd Step: 1 di to many DSD codes

80. Final Step: Register new mappings

81. Exercise 5: Complex time series
Use unstats.un.org/unsd/demodiweb[1-6]
Map/ amend/ publish the time series for;
“Contraceptive prevalence rate”
“Primary completion rate”
“Gender parity index in primary education”
“Seats held by men in national parliament”
“Seats held by women in national parliament”
“Telephone lines”

82. Other issues encountered with generating SDMX from DevInfo
The MDG DSD requires any data point to be uniquely described by the following dimensions;
However, DevInfo allows data to be stored in overlapping time intervals and with multiple sources. These issues need to be resolved to conform to the “uniqueness” required by the MDG DSD.

83. Multiple sources
Allowable in DevInfo but not in the DSD

84. Overlapping time
This issue is only a problem where overlapping periods begin from the same year, as the mapping tool takes the first year in the period as the value for the “Time Period” dimension.

85. Targets in the database
Targets are also an issue when found in the database since they should not be exchanged as observed values

86. Target in database (Example 1)
Sometimes stored as subgroup which can be ignored at the 2nd stage…

87. Target in database (Example 2)
But other times can be found as a time period among observed values…

88. Use of filters at registration
To deal with the issues of;
multiple sources for a given time period,
overlapping time period beginning at the same year;
And targets presented alongside observed values
The mapping tool provides a feature to filter out data from a generated SDMX message associated with specific time periods and source references.

89. Back to mapping…

90. Final Step: Filter by time/ source

91. Final Step: Select source filter

92. Final Step: Select time filter

93. Final Step: Register new mappings

94. Final Step: Complete

95. Exercise 6: Filter time series
Use unstats.un.org/unsd/demodiweb[2-6]
Map/ amend/ publish the time series for;
“Under-five mortality rate”
“Maternal mortality ratio (MMR)”
“Net enrolment ratio in primary education (NER)”
“Orphans primary school enrolment”
“Tuberculosis prevalence rate”
“Proportion of the population using improved sanitation facilities”

96. DSD Maintenance
The mapping and registry tool allows users to edit and delete the DSD as well as upload.
For when the DSD is updated, it is recommended to edit the DSD rather than delete
DSD deletion has the effect of removing all the mappings and subscriptions used for that DSD

97. DSD Maintenance…

98. DSD Header…