1. The process of data collection
Teodora Brandmüller

2. Did things happen according to plan?
Content of my talk
Plan
What to do?
How to do?
Act
How to improve? Do
Do what was planned
Check
Did things happen according to plan?
PDCA (aka the Deming Cycle, Shewhart cycle, or Deming Wheel) is an iterative four-step quality control strategy.
The Shewhart Cycle
PLAN 
establish the objectives and processes necessary to deliver results in accordance with the specifications.
DO 
implement the processes.
CHECK 
monitor and evaluate the processes and results against objectives and Specifications and report the outcome.
ACT 
apply actions to the outcome for necessary improvement. This means reviewing all steps (Plan, Do, Check, Act) and modifying the process to improve it before its next implementation.
Made popular by Dr. W. Edwards Deming, Father of Modern Quality Control, but always referred to by him as the "Shewhart cycle." Later in Deming's career, he modified PDCA to "Plan, Do, Study, Act" (PDSA) so as to better describe his recommendations.
PDCA should be repeatedly implemented, as quickly as possible, in upward spirals that converge on the ultimate goal, each cycle closer than the previous. This approach is based on the understanding that our knowledge and skills are always limited, but improving as we go. Often, key information is unknown, or unknowable. Rather than enter "analysis paralysis" to get it perfect the first time, it is better to be approximately right than exactly wrong. Over time and with better knowledge and skills, PDCA will help define the ideal goal, as well as help get us there.
Velocity of change is a key competitive factor in today's world. PDCA allows for quantum breakthroughs (typical Western approach), as well as Kaizen (typical Eastern approach with continuous improvement); thereby providing the best of both worlds. In this way, PDCA helps ensure the fastest rate of improvement; often a critical success factor.
The power of Deming's concept is in its simplicity. While easy to understand, it is often difficult to accomplish on a on-going basis due to complacency, distractions, loss of focus, lack of commitment, re-assigned priorities, lack of resources, etc. While most claim full knowledge and on-going application, few have in-depth understanding, and even fewer practice PDCA on a consistent basis.

3. The quality check

4. Process of quality controll
validate variable values
Urban Audit Data Quality Control
Update variables and indicators in the UA database
control indicator values
Checked
database
compute indicators using corrected values
control variable values
validate variable values

5. Variable controls Multivariate variable controls
Male Resident Population + Female Resident Population = Total Resident Population
Univariate variable control
sum of the total residents at the sub-city district levels = total residents number at the city level
Multi-variate controls involve more than 1 variable. The data collected enables to control the provided statistics for a specified variable by summing up values of other, related variables. By way of example the values of DE1002V Male Resident Population and DE1003V Female Resident Population were summed up and checked against the provided values of DE1001V Total Resident Population at the city level. The same control was applied at values provided at different spatial levels, i.e. LUZ level, SCD1 & SCD2 and national level.
Some 111 multi-variate variable controls were defined and agreed by Eurostat. The detailed description of the controls and the algorithms applied is presented in annex A5.
Uni-variate controls concern only one variable. Similar to the multi-variate controls, provided variable values were compared with calculated sums, this time not at the same spatial level but across, i.e. values at one spatial level were compared with sums of values provided for the same variable at another spatial level. This is possible due to the hierarchy of the spatial levels in the Urban Audit.
For example, uni-variate control of DE1001V Total Resident Population consists in checking the value provided at City level with the sum of the values for the same variable DE1001V provided at SCD1/2 level for the same location. Some 183 uni-variate variable controls were defined and agreed by Eurostat (see annex A6).

6. Indicator controls
Fixed ranges of accepted values according to common sense
Nationals born abroad as a percentage of total population – Range 0%-20%
Acceptable ranges of indicator values calculated using descriptive statistics
The individual indicator values at each spatial level were checked against fixed control ranges defined by GHK International in the framework of the Urban Audit 2 Lot 1 project. The table defining the fixed ranges was provided by Eurostat (see annex A7).
The individual indicator values at each spatial level were also checked against dynamic control ranges, i.e. the control limits depend on the data itself.
Based on Average and Standard Deviation of indicator values over a specified population, i.e. spatial level (cities, LUZ, SCD1, SCD2 or national level) and groups of countries (EU12 – New Member States and Candidate Countries, EU15 – Old Member States, EU27 – All Countries), the control range was determined by interval +- * . The control consists in checking if the individual indicator value is out of the control range. In this case, the indicator value is not accepted by the control and need to be validated.
To avoid very large control ranges (and possibly negative minimum values) a maximum for the coefficient , i.e. the upper and lower control limit, has been set to the ratio average and standard deviation / . For example, the control limits of SA1026I “Proportion of non-conventional dwellings” for 1999 based on the City values of the group EU15 are set to 0 – 2.73: the average is 1.37 and the standard deviation 1.76, so  =0.78 that is /. 
Exceptions of these setting are the indicators DE1061I “Total population change over 1 year”, DE1062I “Total population change over 5 years” and EC1201I “Annual average change in employment over 5 years” because negative values are possible.
To ensure meaningful descriptive statistics, a minimum number of values in the population is required per reference period to compute average and standard deviation. It has been decided by Eurostat to require a minimum number of 6 values. The control limits are set depending on the number of data points as shown in the table. Indicator controls are not applied if the concerned number of data points is less than 6.

7. Variable controls
Based on Average and Standard Deviation of indicator values over a specified population, i.e. spatial level (cities, LUZ, SCD1, SCD2 or national level) and groups of countries (EU12 – New Member States and Candidate Countries, EU15 – Old Member States, EU27 – All Countries), the control range was determined. The chart presents data for the indicator DE2004I Proportion of nationals born abroad. Data for the 15 EU Memnber States in 2001 is displayed.
The acceptable range marked with orange was calculated on the basis on the average and standard deviation.
Average +- z * standard deviation where z = 2 but limited by Average/ standard deviation
Average 6.2 standard deviation 4.2

8. Variables to validate Results of the variable controls
Results of the indicator controls
Indicators not falling within the “accepted” control range were considered potentially erroneous . All the variables required to compute the indicators were listed to be validated.

9. Assessing lesson learned

10. Conclusions and actions (1)
Variable list needs review
on-going in the “Urban Audit Think Tank”
There are still national or even regional/local definitions applied
The usage of flags and footnotes need to be clarified
Eurostat prepared a proposal which gives clearer instructions

11. Conclusions and actions (2)
Better and clearer definition and labelling
The update of the Methodological Handbook is foreseen
A reference guide containing more detailed meta data is needed
The preparation of a Reference Guide is foreseen

12. Preparatory work

13. Series of Think Tank meetings Major achievements
The list of indicators for the 2006 data collection was reviewed
Special focus on environmental indicators
Integrate the UN Indicators?
Improvement of spatial delimitations (especially LUZ) was discussed
The possibility of adding more cities to the Urban Audit was examined

14. Preparatory work at the Commission and National Statistical Institutes
Updating
the list of indicators and variables
the glossary of variables
Bilateral discussions on
Additional cities
Larger Urban Zones
Kernels

15. Planned timetable Action When
Start grant agreements with 25 Member States
March 2006
Start the data collection
May 2006
Identification of variables
June 2006
Finalise data collection
January 2007
Quality check of the 2006 results
March 2007
Publication of results
April 2007

16. Basic data flow of the Urban Audit
provider
National Urban Audit Co-ordinator
Data is
transmitted according to
the pre-defined
format
Production database of
Eurostat

17. Your turn!

18. Thanks for listening !
Any Questions ?
Any questions?