UPTAP Workshop How Can e-Social Science Promote the Re-Use of Data? Rob Procter National Centre for e-Social Science

UPTAP Workshop The e-Science Vision n “e-Science is about global collaboration in key areas of science and the next generation of infrastructure that will enable it.” (John Taylor, former DG, Research Councils) n That infrastructure is the Grid: “ … a software infrastructure that enables flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions and resources” (Foster, Kesselman and Tuecke) n The Grid is not just an enabler of visionary research, however, but can help researchers in more mundane ways. n But, to be successful, the development of the Grid must be driven by researchers’ needs. n I want to use the opportunity provided by this workshop to gather ideas from you about what those needs are with a specific focus on the (re-)use of data.

UPTAP Workshop NCeSS Overview n Launched in May 2004 to develop and promote UK e-Social Science. n Unified Centre with distributed structure: –Co-ordinating Hub: Manchester & UKDA –Seven research Nodes located across UK –Twelve small projects

UPTAP Workshop NCeSS Overview n Applications of e-Social Science: –Harnessing new kinds of research infrastructure and tools to tackle substantive problems and promote innovation in research methods n Social shaping: –Usability of new infrastructure and tools –Socio-technical factors in their design, uptake and use –Research and policy drivers, impacts

UPTAP Workshop Hub Social Shaping NCeSS 2006 Tools CQeSS MoSeS PolicyGrid Disclosure Risk Assessment CeSDeMIDE GeSRM Intelligent Simulation MiMeG HeadTalk Analysis Infrastructure and services Research methods OeSS DReSS AGN enabled interviews Learning Disabilities Entangled Data Data chronicles Replayer Grid-enabled data collection Data GeoVUE GeODE

UPTAP Workshop Today’s Research Infrastructure n Heterogeneous resources with poor inter- operability and complex administrative arrangements. HPC Analysis Data archive Analysis Study Experiment HPC Researcher Computing Data archive n Doesn’t scale well and makes re-use and sharing of data and other research resources difficult.

UPTAP Workshop Grid-Enabled Research Infrastructure Social scientist Grid Middle- ware Storage Computing Analysis Experiment HPC Grid middleware manages the interactions between users, and heterogeneous and distributed resources, providing seamless integration of data, analytic tools and compute resources. Data archive Study

UPTAP Workshop The Grid Dissected n Tools to support collaboration between distributed researchers. n Computational Grids for scalable, high- performance computation. n Data Grids for accessing and integrating heterogeneous datasets. n Sensor Grids for collecting real-time data.

UPTAP Workshop Research and Policy Drivers Ageing population Migration Globalisation Childhood development Census and population surveys Administrative data Longitudinal surveys Socio- medical data Business and economic data International macro/micro data       

UPTAP Workshop Research and Policy Drivers n The range of research resources on offer to the social science community has never been greater. n These include not only traditional research datasets, but new kinds of social data. n However, the often highly distributed and heterogeneous character of these datasets makes it difficult to exploit them to their full potential.

UPTAP Workshop Research and Policy Drivers n The data deluge in social sciences: –WWW archive currently contains 55 billion Web pages or 2 petabytes (2x10 15 ) of data and is growing at the rate of 20 terabytes (20x10 12 ) per month n Administrative and transactional data is generated on increasing scale as by product of our everyday activities: –This data is complex and multi-dimensional

UPTAP Workshop Data Grids for Social Science n Data Grids are designed to provide unimpeded and integrated use of distributed, heterogeneous, autonomous data resources. n Grid enabling a dataset creates new opportunities for (re-)use: –enables users to integrate it with other datasets –makes it possible to analyse the dataset using techniques that require the kind of computational power that is only feasible using the Grid (e.g., more complex models, more data points) –standardisation of procedures and mechanisms used to access and update the dataset increase its shareability –Automated analyses (i.e., analyses can be re-run automatically when databases are updated)

UPTAP Workshop An Example Data Linkage Problem n Many research questions require combination of data from multiple geo-referenced datasets: –E.g., Linking post coded data to census geography n Conversion of data relating to different geographies to a common target geography is –A complex time consuming task –Requires a range of data handling/processing skills –A major barrier to use! n The data conversion process requires users to perform the following generic tasks: –Extract and download data in different formats from a number of databases using different interfaces –Convert each dataset to the desired target geography using geographical conversion tables –Combine the converted sets into a single dataset for analysis n These generic tasks can be automated.

UPTAP Workshop A Solution: ConvertGrid n ConvertGrid provides access to 225 UK-wide geography conversion tables between census, electoral, administrative, postal, health and statistical geographies derived from the AFPD. n Facility to convert a researcher’s data from one set of geographical units to another (e.g., from postcode geography to heath geography). n Extensible system - further conversion tables from any source can be incorporated.

UPTAP Workshop ConvertGrid – Data Visualisation Interface n Relationship between average house price sales (Experian) and percentage of year olds entering university (Neighbourhood Statistics & Census aggregate statistics). n Contact Keith Cole for more High average house price sales but low participation rates Low average house price sales but high participation rates Ten minutes from start to finish

UPTAP Workshop Supporting the Research Lifecycle Share results and conclusions and discuss with collaborators Explore datasets and determine suitability Analyse results and compare with hypothesis Review literature and generate hypothesis Write papers Build models and execute them Publish papers Find datasets related to proposed area of work

UPTAP Workshop Increasing (Re-)Use of Social Data n Removing barriers to more effective use of existing social data collections: –Data providers (e.g., ONS, data archives) –Data users n Many researchers are both generators and users of data: –Preparation of data for submission to data archives is not well rewarded so re-use suffers n Removing barriers to use of new kinds of social data: –Privacy and confidentiality of personal data

UPTAP Workshop The Data Provider Perspective n Preparation procedures: –Cleaning the data –Generating derived variables –Re-weighting –Adding metadata –Writing user documentation n Maintenance: –Managing changes in sampling frames, definitions, variables and questionnaire over time –Re-weighting n User support: –Handling queries from users about concepts, meaning and linking waves

UPTAP Workshop The Data User Perspective n Discovering appropriate data: –Determining what can be done with the data and how. n Accessing the data: –Are existing provisions, such as VMDLs, for access to confidential data adequate? n Understanding how the data has been used to generate answers to other research questions: –Provenance of results, links to publications –Re-running statistical models, comparing results n Ease and of use and quality of documentation: –User manuals

UPTAP Workshop The Data User Perspective n Data preparation: –Selecting variables –Linking waves –Linking data sets n Performing and possibly repeating analysis with different data. n Interpreting and visualising results. n Supporting the research lifecycle. n Collaboration with other users and with data providers.

UPTAP Workshop Contacting NCeSS and Getting Involved n n –Join our list: –Participate in events: Agenda setting workshop on combining and sharing data, January 22 nd -23 rd, Manchester Annual conference