Deborah Mitchell The Australian National University Transforming the Australian Social Science Data Archive

Introduction to ASSDA Changing environment - cultural attitudes to data sharing - growth and diversity in data creation - technological change Response to change: the new ASSDA The distributed data Archive as a grid environment Presentation Overview

ASSDA was set up in 1981 at ANU in Canberra Archive is housed in the Research School of Social Sciences The Archive holds some 1500 data sets, core holdings being survey based data Most notable holdings are national election studies; public opinion polls; social attitudes surveys The Archive also acts as a custodian for data from SE Asia, New Zealand and smaller Pacific nations ASSDA in Brief

For most of its history, ASSDA operated in a highly centralised fashion serving a small, fairly tight knit community of users - who were also willing depositors Up until 1995 the Archive also acted as the distribution point to the academic community for official data collections of the Australian Bureau Statistics (ABS) Complementarity between the holdings and functions of ASSDA: the general ABS data underpinning the specialised research surveys carried out by the academic community Operating environment

1. Cultural shift in attitudes towards data -ABS withdrew open, free-of-charge access to its data and ‘commercialised’ its data distribution -This sent a powerful signal that data/information is valuable; academic community responded to this signal by hoarding its research collections In the period , survey data deposition declined markedly. What changed?

2. Growth and diversity in data collection -As the ABS gradually tightened access to official data and ‘priced’ itself out of the academic market, the academic community significantly stepped up its own primary data collection efforts -Qualitative methods were being embraced and creating large numbers of small area studies and new forms of data - linguistic; web pages; visual images were being shaped into digitised databases The Archive was being called on to manage increasing amounts of data, in unfamiliar formats What changed…ctd

3. Technological change -Memory became cheap! -Proliferation of software & other analytical tools to deal with non-quantitative data - Advent of grid-computing and e-Research User expectations of what the Archive could/should be doing grew rapidly. What changed…ctd

1.Cultural change Re-establish a culture of data sharing within the academic community; open up the governance of the Archive to the research community. 2. Growth & diversification in data Look to a means of growing the Archive’s capacity and skill base in terms of handling both greater volume and type of data. 3. Technological change Harness new technologies, software tools in a staged a manageable fashion, with particular attention to user demands. The ASSDA Response

Moving to a distributed archive solution is not an ‘obvious’ solution in the Australian context, as economies of scale still favour the original centralised model that we started with in the 1980s. High-speed broad-band networks servicing the academic community across the Australian continent also favour a centralised solution. However, we have decided to move to a distributed archive form in order to solve several problems: Diversity in forms of data; Growth in volume of data and knowledge about where these data are located; and To deal with data hoarding. ASSDA Distributed Archive

NESSTARWebViewNESSTARWebView ASSDA -ANU ASSDA -UQ ASSDA -UNSW The ASSDA Grid Solution

In summary: The Archive will become a series of ‘nodes’ each of which will specialise in a particular area, determined by the research strength of each host institution. In the first round of development, nodes will be established at the U. Queensland [qualitative data] and UNSW [administrative data]. As shown in the previous slide, we will be leaning quite heavily on the capacities of the Nesstar software to deliver one interface to users. The ASSDA Plan

The ANU-UQ-UNSW network will be set-up in 2005 to function in a grid fashion,with middleware designed at ANU to load-share catalogue enquiries and Nesstar on-line analyses between the three servers. We expect that at each node, specialised analysis tools will be developed for data analysis, beyond Nesstar capacity. Over time, we aim to establish at least one node in each state that will broaden and deepen the ASSDA holdings in a significant fashion. The ASSDA Plan … ctd