1. Nordisk Statistikermøde i København 11.-14. august 2010 The archive statistical method 1960-2010 50 years - A Summary by Svein Nordbotten 8/11/20101Svein Nordbotten

2. 8/11/2010Svein Nordbotten2 The situation 50 years ago Separate statistical organizational units for limited subject matter domains Few possibilities to link data within and between established domains Reuse of micro data expensive and time consuming Creating longitudinal micro data sets rarely possible Published statistics followed a narrow and restricted tabulation plan Analysis limited to aggregated macro data Statisticians were experts in their subject matter domains Little exchange of experience and information among different groups

3. 8/11/2010Svein Nordbotten3 Two important events in the 1950s Electronic, programmed computers made processing of large data sets more effective for administrative and statistical purposes. Electronic recording made establishing, maintaining and sharing comprehensive population/object registers possible.

4. 8/11/2010Svein Nordbotten4 The information technology The UNIVAC 1 computer entered the official statistical scene in the US Census Bureau in the early 1950’s, and when the processing of the 1960 censuses were due, a number of NSIs were prepared with their own computers. When the information technology later merged with communication technology, it changed the time and space dimensions for statistical processing. The new technical potensial inspired to think about all activities of an NSI as interrelated parts of a production system in contrast to the traditional separate statistical processes, and the term statistical production became common.

5. 8/11/2010Svein Nordbotten5 Administrative registers Local object identifiers had been used for a long time in many admistrative processes. These specialized systems were expensive to maintain, represented a vast duplication of efforts and were of limited usability as statistical data sources. The efficiency of shared, public object identifiers were obvious. About 1950 the first steps to establish general, permanent, unique object identifier registers for the economic units as well as for the population were taken in the Nordic countries with active participation by statisticians.

6. 8/11/2010Svein Nordbotten6 Statistical ideas in the 1950s Unique and permanent identifiers permitted compilation of the ’life lines’ of statistical objects including interobject events by data linkage. Models for statistical micro systems named statistical archive systems, statistical file systems, register-based statistical systems, etc. were discussed in international fora from about 1960. The next slides illustrate the popular version of ideas discussed. Ten years later the generic version became known as data base management systems.

7. 8/11/2010Svein Nordbotten7 6/10/2015Svein Nordbotten7 Objects Time Location The 3 basic concepts

8. 8/11/2010Svein Nordbotten8 Objects Attributes Time Census attributes Census day Human population Archive data container

9. 8/11/2010Svein Nordbotten9 Time Objects Time plane Object plane Attribute plane Attributes The 3-dimensional space model

10. Object:B Time: t-1 Object type A type A Time t Object type A Time t Object: C Time: t-1 Object: C Time: t OOOOO Object A: Time: t-2 Object: C Time: t-2 Object : A Time: t Object: B Time: t Object: B Time: t-2 Object: A Time: t-1 Link H Link V 8/11/201010Svein Nordbotten Latitudinal and longitudinal integration

11. 8/11/2010Svein Nordbotten11 An example

12. 8/11/2010Svein Nordbotten12 Summary of the 50 year old visions Comprehensive official register systems General utilization of systems Fast and high capacity IT Extensive and userfriendly communication Software for data processing, saving, retrieval and communication Data collection independent of specific usage Use of communication network for collection Functional organization Wider statistical coverage by more intensive reuse of data Statistics on-demand Analytical interest in and tools for exploiting the new possibilities Legislation regulating the use of micro data

13. 8/11/2010Svein Nordbotten13 Utilization of public identification systems Population register systems in general use Economic unit register systems in general use by administrative organizations Register systems for geographical location of properties established and in use

14. 8/11/2010Svein Nordbotten14 Sample of present applications Censuses (population, location registers) Demographics (demographic registration) Foreign trade (custom data) Income (taxation data) Social statistics (registration of services) Employment (unemployment registration) Education (registration of students) Health (medical registrations) Crime ( Police, court, prison registrations) Business (Business registers)

15. 8/11/2010Svein Nordbotten15 Methodological challenges The present intensive utilization of administrative data in official statistics has also required work on methods in several fields: 1.Electronic processes for collecting administrative data 2.Quality control of data received from external administrative sources 3.Integrating data from different statistical and administrative sources 4.Evaluating quality of integrated data sets 5.Editing data using associated administrative background data 6.Estimating population parameters using administrative background data 7.Confidentiality and security in systems based on administrative data 8.Statistics on demand from systems based on administrative data 9.Evaluation of costs and benefits from using administrative data

16. 8/11/2010Svein Nordbotten16 Privacy issues Statistical legislation Privacy legislation Misuse of information Value of knowledge Regulatory use and research Solutions? A political question

17. AD1 ADM 1 AD2 ADM 2 SD SP NSI Information privacy law Statistical law Data collection Data products 8/11/201017Svein Nordbotten

18. 8/11/2010Svein Nordbotten18 Anticipations/realizations Comprehensive official register systems…………………………………………….... Yes General utilization of systems………………………………………………………………. Yes Fast and high capacity IT………………………………………………………………………. Yes Extensive and userfriendly communication…………………………………………… Yes Software for data processing, saving, retrieval and communication……… Yes Data collection independent of specific usage…………………………………….. ? Use of communication network for collection......................................... Yes Functional organization……………………………………………………………………….. Yes? Wider statistical coverage by more intensive reuse of data…………………. Yes Statistics on-demand…………………………………………………………………………… ? Analytical interest in and tools for exploiting the new possibilities……… Yes? Legislation regulating the use of micro data………………………………………… Yes?

19. 8/11/2010Svein Nordbotten19 What next? 1.Electronically observed data 2.Mobile use of statistics Require legislative, technical and methodoligical preparations.

20. 8/11/2010Svein Nordbotten20 Thank you all for your attention! I look forward to seeing you all in another 50 years.