Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest Preservation for Ongoing Accessibility: research group Professor Ross Harvey Dr Bob Pymm Dr Anne Lloyd Geoff Fellows Jake Wallis

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest Pandora - NLA solution to website preservation Archive of over 1.7 terabytes of data selective - identifies specific sites for harvest and gains permission to archive

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest Internet Archive - Automated Harvests ‘the web’ issues? – cost – reliability of the crawl eg deep web

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest.au Harvest by Internet Archive first ran producing 6.9 terabytes of data, 185 million unique files Issues? – difficulties with certain file types – password-protected sites – difficulty in accessing the ‘deep’ web

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest.au Harvest September 2006 – more sophisticated crawl 19 terabytes of data, 596 million files predominant dataset for POA group

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest Research potential? digital preservation Australian digital culture

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest 3 broad questions What are the contents of the harvests? How can access be provided to this content? What is the value of the domain harvests in relation to the NLA’s overall web preservation interests?

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest Blogs low skill threshold technology as barometer of engagement social space catalyst for online community a new and important collecting point for digital cultural heritage

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest Archiving and preserving blogs how to identify Australian specific material? what to capture – selection criteria? – linked material? frequency of capture to ensure accurate representation provision of access to harvested blog content

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest Aspirations a conceptual framework for studies in digital anthropology a broadening of voices within the Australian public sphere

Separating the wheat from the chaff: Identifying key elements in the NLA.au domain harvest Questions/comments?