1. ETANA-ADD: An Interactive Tool for Integrating Archaeological DL Collections JCDL 2006, Chapel Hill, NC June 13, 2006 Naga Srinivas Vemuri, Rao Shen, Sameer Tupe, Weiguo Fan, Edward A. Fox fox@vt.edu http://fox.cs.vt.edu

2. Acknowledgements (Selected) Sponsors: NSF grant ITR-0325579, ASOR, CWRU, ETANA, Vanderbilt U., Virginia Tech VT Students: Vidhya Vijayaraghavan, other DLRL members Others: Umm el-Jimal Dig Team

3. Acknowledgements (Selected) Karen Borstad, MPP Giorgio Buccellati, UCLA Douglas Clark, Walla Walla College Joanne Eustis, CWRU Nick Fischio, CWRU Israel Finkelstein, Tel-Aviv University Paul Gherman, Vanderbilt U. Andrew Graham, U. Toronto Tim Harrison, U. Toronto Larry Herr, Canadian University College Christopher Holland, LRP Paul Jacobs, Mississippi State U. Douglas Knight, Vanderbilt U. Stan LaBianca, Andrews U. David McCreery, Willamette U. Eric Meyers, Duke U. Adam Porter, Illinois College Jack Sasson, Vanderbilt U. Tom Schaub, Indiana U. of Penn. Randall Younker, Andrews U.

4. Outline  Introduction  Related Work  ETANA-ADD Tool  Conclusions

5. Introduction  ETANA, ETANA-DL, 5S  What are the issues involved in integrating new collections into a DL, with evolving metadata schema (i.e., bottom up schema evolution)?  Can we partially automate the process of integrating new collections in such situations?

6. ETANA-DL Heterogeneity: 8 archaeological sites, 13 different artifact types Example artifact types: Bone, Burial, Figurine, Locus, Pottery, Seed, etc. Union services: Multidimensional Browsing, Searching, Recommendation, Annotation, etc.

7. ETANA-DL (Cont.)  Individual (archaeological) site approach Local conventions for metadata Custom built services  ETANA-DL Provides union services across sites A global schema based on incremental approach

8. The Mapping Process in ETANA  Mapping Process: Global schema defines collections (metadata) in the system using an incremental approach.  Adding a new artifact collection if artifact type is already defined, perform mapping if artifact type is not defined, then extend global schema and perform mapping

9. The Whole Integration Process  Conversion process: custom DB to XML format Needs to identify metadata elements in DB Results in local XML data, local XML schema  Mapping process Needs to perform schema mapping Results in global schema extension, and the evolution of a new global collection  Integration process New site to be “published” as OAI Provider OAI harvesting results in integration of the new collection.

10. The Integration Problem  Problem: The integration process requires both technical and domain expertise.  Propose: Partially automate the process to minimize the need for technical skills  Solution: ETANA-ADD Tool

11. Outline  Introduction  Related Work  ETANA-ADD Tool  Conclusions

12. Related Work  Gatherer: A tool used in Greenstone for adding new collections. Tightly coupled with Greenstone No knowledge of its ability to handle evolving schema and its content  Database to OAI Provider: OAICat and OAI PMH2 Perl Doesn’t accommodate mapping process

13. Related Work (Cont.)  OCHRE proposed archaeoML to define DL collections Doesn’t automate integration process Ability to handle heterogeneous data is not known  Altova MapForce Doesn’t support incremental mapping

14. Outline  Introduction  Related Work  ETANA-ADD Tool  Conclusions

15. ETANA-ADD Tool  An interactive tool for end users Partially automates the integration process Minimizes the need for technical skills Reuses existing tools to some extent, by providing easy GUI wrapper on top of them

16. ETANA-ADD Tool (Cont.)  The process flow involved while using the tool: DB2XMLSchema Mapper OAI XML File Data Provider

17. An Integration Scenario  Adding burial artifacts collection to ETANA DL Perform DB2XML process using ETANA-ADD Perform Schema Mapping Publish Burial Collection as OAI Data Provider

18. Initial Screen with Umm el-Jimal Database Open

19. Tables corresponding to Burial artifact selected

20. Performing join on tables for burial artifact

21. DB2XML Process Complete

22. Invoking Schema Mapper

23. Opening Global Schema

24. Performing Mapping Process

25. Extending Global Schema to Integrate Burial Artifact

26. Mapping Complete, Generating Global XML Collection

27. Complete Global XML Generation, Publishing as OAI Provider

28. Publishing as OAI Provider

29. Results  Integrated Ummm el-Jimal site with the help of ETANA-ADD Bone, Burial, Locus, Miscellaneous Artifact, Pottery, Pottery Bucket.  No additional code written  A comparison with earlier integrated site, Megiddo (7 artifact collections)

30. Results (Cont.) Umm el- Jimal Megiddo Additional LOC Required 0 1350 Human Hours 2 20

31. Outline  Introduction  Related Work  ETANA-ADD Tool  Conclusions

32. Conclusions  Target users: Administrators handling archaeology data (to be invited in fall for usability studies)  Developed ETANA-ADD to minimize technical expertise in integrating new archaeological collections  Willing to share our software, which may be applicable to other domains with similar problems (i.e., with evolving global schema)

33. References  Brainbridge, D., Thompson, J., and Witten, I. H. Assembling and enriching digital library collections. In Proc. JCDL 2003: 323-334.  Raghavan, A., Vemuri, N. S., Shen, R., Gonçalves, M.A., Fan, W. and Fox, E.A. Incremental, Semi- automatic, Mapping-Based Integration of Heterogeneous Collections into Archaeological Digital Libraries: Megiddo Case Study. In Proc. ECDL 2005: 139-150.  Ravindranathan, U., Shen, R., Gonçalves, M.A., Fan, W., Fox, E. A., Flanagan, J.W., ETANA-DL: a digital library for integrating heterogeneous archaeological data. In Proc. JCDL 2004: 76-77.  Suleman, H. Open Digital Libraries, Ph.D. Dissertation, Dept. Comp. Sci., Virginia Tech, http://scholar.lib.vt.edu/theses/available/etd- 11222002-155624, 2002.

34. Questions/Comments ?