1. www.RoperCenter.uconn.edu1 Upgrading ABC News/Washington Post Data Collections Using DDI and Legacy Databases Marc Maynard The Roper Center for Public Opinion Research University of Connecticut IASSIST Conference 2005, Edinburgh, Scotland

2. www.RoperCenter.uconn.edu2 Upgrading Data Collections Introduction Background Scope Challenges & Opportunities Prototype System Summary

3. www.RoperCenter.uconn.edu3 The Roper Center Archives Public opinion data archive established in 1946 Commercial and academic surveys from 1936-present The Archives house ~8,000 US and ~7,000 non-US surveys  including data files & documentation ABC News/Washington Post Survey Collection  Over 850 surveys 1979-present

4. www.RoperCenter.uconn.edu4 Background: Metadata Integration Catalog of Holdings –study level –15,000 records –only studies for which raw data is housed at the Center iPOLL Databank –variable level –nearly 500,000 records –includes studies for which data is not housed at the Center

5. www.RoperCenter.uconn.edu5 Background: Metadata Integration External Review (2001) Overall Integrated Vision (IASSIST 2002) DDI – Archive Catalog Mapping (IASSIST 2003) –Study and File Level Integration (Sections 2 & 3) iPOLL  Archive Catalog Links (2003-2004) Enhance Question/Variable Metadata (IASSIST 2005)

6. www.RoperCenter.uconn.edu6 Background: Prototype Project ABC and the Post want to easily access and analyze all their survey data SPSS system files for post-1997 surveys exist Pre-1998 studies are a hodge-podge of available ASCII data, documentation and survey reports ABC experimented with various alternative strategies Determined that the major cost factor would be variable and response labeling

7. www.RoperCenter.uconn.edu7 Scope >600 ABC/WP surveys, 1979-1997  More than 16,000 questions in the iPOLL system  Fairly consistent documentation and data structure  All ASCII data files Average about 35 variables per study  Not including standard socio-demographic variables Employ a prioritized phased approach Focus on joint monthly surveys (216 studies)

8. www.RoperCenter.uconn.edu8 Challenges 1.iPOLL includes only surveys of US adult population 2.iPOLL does not store standard socio- demographic variables 3.Published results are source for many items 4.iPOLL does not store enough metadata on the variable level

9. www.RoperCenter.uconn.edu9 Opportunities Enhance metadata available in iPOLL Repurpose iPOLL’s store of question text and response categories Capitalize on: –the fact that response categories are stored as individual items –Linkages between question-level information and existing data files

10. www.RoperCenter.uconn.edu10 Addressing the Challenges 1.iPOLL includes only surveys of US adult population State/Local surveys are lower priorities 2.iPOLL does not store standard socio- demographic variables Add standard demogs menu to system 3.Published results are source for many items Must allow for modifications to the variables 4.iPOLL does not store enough metadata on the variable level Extend iPOLL DataBank with DDI elements

11. www.RoperCenter.uconn.edu11 Mapping Scheme - Sec. 4 Question/Variable DDI ElementDatabase Field 4.3NameVarName 4.3.8.2QstnLitQstn_txt 4.3.1RecSegNo 4.3.1StartPos - EndPosLocation 4.3.1WidthvarWidth 4.3.23varFormat Response Categories DDI ElementDatabase Field 4.3.18.2lablResp_Txt 4.3.18.1catValuResp_Code 4.3.18Missing

12. www.RoperCenter.uconn.edu12 File Preparation iPOLL SPSS: Enhanced variable- level metadata iPOLL (q/v) Project Application ASCII Data File SPSS Portable File SPSS Syntax File Archive Catalog Standard Demogs

13. www.RoperCenter.uconn.edu13 Application Requirements Edit and add missing metadata to each variable –Variable names, location, type Review and edit response category coding Select and add standard socio-demographic variables Specify any recodes within variables or to new variables Handle string, as well as numeric, value labeling and recoding Generate SPSS syntax file to include study metadata, creation date and data file path and structure

14. www.RoperCenter.uconn.edu14 Prototype System

15. www.RoperCenter.uconn.edu15

16. www.RoperCenter.uconn.edu16

17. www.RoperCenter.uconn.edu17

18. www.RoperCenter.uconn.edu18

19. www.RoperCenter.uconn.edu19 Summary Continuation of metadata enhancement and integration efforts begun in 2001 Will provide practical feedback and suggestions for extending the capabilities of iPOLL Promising beginning for expanding coverage to other data collections

20. www.RoperCenter.uconn.edu20 iPOLL Databank can be found at: http://www.ropercenter.uconn.edu/ipoll.html Email: Marc.Maynard@UConn.edu