1. From Survey Design to Hackfest:
Reflections on Research-in-Practice for Born-Digital Access
Rachel Appel, Special Collections, Bryn Mawr College Library & Information Technology Services
Alison Clemens, Beinecke Rare Book & Manuscript Library, Yale University
Wendy Hagenmaier, Georgia Tech Archives
Jessica Meyerson, Briscoe Center for American History, University of Texas
[Alison]
Archival best practices for processing and preserving born-digital materials have developed over the last decade. However, there are no established best practices for providing public/research access to born-digital materials that both scales to match both the volume of born-digital material and acknowledges organizational capacity. We agreed that only by documenting existing practices can we advance the professional discourse around establishing best practices for born-digital access. Therefore, our first phase of research included survey design and distribution, followed by semi-structured interviews. The qualitative and quantitative data were examined to pinpoint aspects of born-digital access that participants classified as gaps and deemed highly important, such as gaps in tools and systems; gaps in resource allocation and advocacy; gaps in archivist skillsets; gaps in understanding users; and gaps in research and policy.
The second phase of research included developing a collaborative hackfest at SAA 2015, where each group was given a specific gap area (as outlined above) and tasked with working on a proposal that outlined possible solutions/strategies for addressing those gaps.
The research outcome therefore includes three datasets: survey responses, interviews, and hackfest documents to be utilized for practitioner and researcher discourse on the topic of born-digital access. The research outcome also includes research team reflections on the research-in-practice within the archives field.

2. Introduction
Working toward best practices for access to born-digital materials that:
Scale to match the volume of born-digital material
Acknowledge organizational capacity
Advancing the professional discourse around best practices for born-digital access
[Alison]
Archival best practices for processing and preserving born-digital materials
However, there are no established best practices for providing access to born-digital materials that
Scale to match the volume of born-digital material
Acknowledge organizational capacity
Documenting existing practices will advance the professional discourse around best practices for born-digital access

3. Phase I
Our first phase of research included survey design and distribution, followed by semi-structured interviews
Survey respondents
129 complete (292 initiated)
Interviewees
20 qualitative follow-up interviews
Recruited from survey respondents
Preliminary report at:
[Alison]
Our first phase of research included survey design and distribution, followed by semi-structured interviews. The qualitative and quantitative data were examined to pinpoint aspects of born-digital access that participants classified as gaps and deemed highly important, such as gaps in tools and systems; gaps in resource allocation and advocacy; gaps in archivist skillsets; gaps in understanding users; and gaps in research and policy.
In order to map the current landscape of born-digital access, we designed an iterative, participatory, mixed-methods study. Phase I of the study included a survey of cultural heritage practitioners and in-depth interviews, conducted from summer 2014 through spring The qualitative and quantitative data were examined to pinpoint significant findings, including aspects of born-digital access that participants classified as highly important gaps: gaps in tools and systems; in resource allocation and advocacy; in archivist skillsets; in understanding users; and in research and policy. We also identified several areas associated with high rates of planning activities, including reading room, remote, and online access; metadata for access and processing; and methods for the creation of copies and images. Our paper is grounded in a rich and extensive set of anonymized data from the surveys and interviews (that we hope to make available via an open access repository) and encourages other scholars to reuse the data and extend the research.

4. Phase I Identifying important gaps Methods: R and pandas
Understanding users
Advocating for born-digital access
Using agile techniques for facilitating born-digital access
Creating training opportunities, or bootcamps, for born-digital access
Methods: R and pandas
Initial impact of preliminary report (bit.ly/hackbdaccess-report)
Creating our SAA 2015 Born-Digital Access Hackfest
[Alison]
We conducted data analysis to identify gaps in born-digital access. The analysis began in R, where some initial frequency plots were created. R is popular in the social sciences; however, it does have its own syntax to learn
The plots included in preliminary report were created using pandas (a thin python wrapper for statistical/data analysis tools and plotting tools including matplotlib)
Through this data analysis, we identified the following gaps:
Understanding users
Advocating for born-digital access
Using agile techniques for facilitating born-digital access
And creating training opportunities, or bootcamps, for born-digital access
We also examined the impact of our preliminary report
Bitly reported that the report got 2,046 clicks
This shows a high level of engagement, and indicates to us that it’s useful to share work in progress
Count number of users + comments?

5. Phase II
Born-Digital Access Hackfest: Collaborative Solution-Building for Current Challenges, SAA 2015 (
Hackfest Team topics (August January 2016):
Understanding Users for Access
Advocacy for Access
Agile Methods for Access
Archivist Bootcamp for Access
[Wendy:]
We used our findings from Phase I to design a hands-on Born-Digital Access Hackfest session at the Society of American Archivists Annual Meeting in August 2015 ( Our goal with the Hackfest was to get a large group involved in analyzing our data and tackling access challenges head-on by developing practical proposals for access solutions designed to mature beyond the Annual Meeting. During the Hackfest, we gave each of four teams a specific topic area, distilled from the gaps, plans, and significant findings identified in Phase I: understanding users, advocacy, agile methods, and an archivist training bootcamp. We tasked the teams with working on a proposal that outlined possible solutions and strategies for addressing those gaps.
50 people attended the session.
From August 2015 through January 2016, the Hackfest Teams developed polished two-page executive summary style proposals (similar to what you might create for a grant proposal) for collaborative projects that would confront current obstacles and have significant practical impact on archivists who are working to providing access to born-digital materials. The aim of the Hackfest was thus to prototype an innovative model of using research results to kickstart collaborative progress towards the future of archival practice. This Hackfest experiment involved lots of people and moving parts, so we worked hard to create straightforward objectives for the team, with clear structures for achieving each objective.
We established the following criteria for the collaborative projects the teams would propose:
short or long-term project, usually involving archivists from multiple institutions
external funding could be made available for the project
goal of the project would be to produce a tangible deliverable (e.g. publication/report, tool, website, database, data set, conference/event) that could help a wide range of archivists to provide better access to born-digital material
focus on ways to improve the practical, day-to-day access process for practitioners (rather than theoretical explorations)

6. Phase II Team roles: Researcher Leader Notetakers Team deliverables:
Notes
Proposal
SAA ERS blog post (to introduce and publish proposal):
[Wendy:]
Each Team had the following roles:
1 RESEARCHER: one of us, from the Mapping the Landscape of Born-Digital Access research project, who got the Hackfest Team started and collaborated with the LEADER to complete the Proposal after SAA.
1 LEADER: a member of the Hackfest Team who volunteered to lead the discussion and to complete the Proposal after SAA. Their primary responsibility was to ensure that the group completed their goals in the time allotted for each portion of the Hackfest. We asked the LEADER to be comfortable dedicating at least 6 hours between August and December to polishing their team’s proposal and writing a short blog post for the SAA Electronic Records Section blog.
1 or 2 NOTETAKERS: one or two members of the Hackfest Team who volunteered to take notes during the SAA Hackfest session. These notes were the primary documentation from the in-person Hackfest session and served as the basis from which the group continued its work to complete their proposal.
During the in-person session, the teams had 45 minutes to sketch out a framework for their proposal. We provided each team with anonymized excerpts from our research data, as well as a list of additional resources that could be helpful for their topic.
Then, between August and January, the teams used their notes from the in-person session and any additional resources desired to writ the proposals and a blog post for the SAA Electronic Records Section blog. We initially tried using Google Groups to facilitate the work of the teams, but plus google docs seemed to be more convenient for everyone, so we dropped the google groups. We provided templates for the proposals and blog posts. Our hope was that these project proposals could be used to organize actual collaborative projects in the near future, led by team members or by others in the archival community. The proposals offered a new and exciting way to kickstart collaborative progress towards born-digital access.

7. Phase II Bootcamp for Born-Digital Access Group
The bootcamp for born-digital access group, led by Dan Johnson of University of Iowa, has met 6 times since last SAA, and has conducted an environmental scan of born-digital access educational opportunities
The group has also brainstormed about curriculum and scheduling for a born-digital access bootcamp training experience
Now, the group is working to identify an opportunity to pilot the bootcamp
[Alison]
It’s worthwhile to share works in progress, and that engagement and vulnerability resonates with people. However, in terms of following through with casual projects, this is what happened
The bootcamp for born-digital access group, led by Dan Johnson of University of Iowa, has met 6 times since last SAA, and has conducted an environmental scan of born-digital access educational opportunities
The group has also brainstormed about curriculum and scheduling for a born-digital access bootcamp training experience
Now, the group is working to identify an opportunity to pilot the bootcamp

8. Google Folder Orientation Slide
Google Folder contains:
Preliminary report
Sample hackfest team folder
How to Design a Data-Driven Hackfest
Our hackfest slides
This presentation
[Jessica]
In order to facilitate access to our research outcomes and our tools, we’ve created a Google folder, with the link bit.ly/researchbdaccess.
This Google folder contains:
A copy of our preliminary report
A sample hackfest team folder, with example documents used during Phase I of the hackfest (i.e. during the SAA session)
A document outlining how to design and run a data-driven hackfest
Our hackfest slides
A copy of this presentation

9. Retrospective Analysis
Overlapping/complementary literatures:
Business/organizational studies
Sociology
Information studies
Key methods/theories/concepts:
Participatory Action Research
Community Impact
Research & Development
We gained experience and skills but lacked a theoretical framework for understanding the impact of our work:
Within the archival community
On ourselves, as reflexive practitioners
Phase III - mapping key methodological/theoretical/conceptual tools-to-think-with back to the work we completed
[Jessica]
Reflecting on this process and its significant evolution in terms of scope, activities and output - we certainly felt that our work allowed us to gain invaluable experience and skills: survey design, interviewing, data analysis (Dedoose, Qualtrics, pandas, R), group facilitation techniques and effective (or not so effective) methods for sustained community engagement
However, what we really felt was lacking was a theoretical lens/framework/scaffolding from which to hang or make sense of the work we had done, and to derive insights from the results of that work (inclusive of both Phases I and II)
It was clear that a Phase III was needed - a Phase where we explored similar cases/research-in-practice within and without the information studies literature and attempted to mapped the findings back to our work - we needed more sophisticated conceptual tools to articulate and parse the meaning of our research effort
Complementary Literatures:
Business/organizational studies:
Sociology
Information studies
Key theoretical/conceptual tools:
Participatory action research
Sarah Maiter, Laura Simich, Nora Jacobson & Julie Wise (2008) Reciprocity: An ethic for community-based participatory action research, Action Research, 6:3,
“Reciprocity is not only necessary to accomplish research in an ethical manner, but it is also illuminating, since the process of negotiating priorities and learning what the study participants expect to obtain from cooperating with researchers reveals valuable cultural knowledge”
Community impact: This is where we see the utility of discussing PAR and CI together in the context of retrospectively understanding the last two years of our work. We can apply PAR to the analysis of what we were aiming to do, but there are ways to address many of the difficulties that we encountered by looking to CI. At the same time, CI raises questions for me about the extent to which our community, the archives community, has some of the core components required in order to support collaboration around complex problems.
Liz Weaver (2016) Possible: Transformational change in collective impact, Community Development, 47:2, , DOI: /
6 conditions for collective impact:
Practice system leadership
Were we able to identify system leadership throughout this process? Were we system leaders? Did the hackfest serve to identify system leaders?
Embrace a framework
If we look at the hackfest as a framework for generating a concrete project to get conversations about broader professional issues started - how do we evolve that into a framework that is inclusive of existing project-based frameworks?
Assess community readiness
Understanding the current state of the field - understanding the current state is key to knowing where to begin in any collective impact effort
Focus on data and measurement
“What can be measured, can be managed” - the question of measurement in archival practice seems to be of critical importance, both in terms of:
Finding our place and telling our story within the broader context of a big data, data analytics-focused environment; it is a question of control, of resources - we are in the best position to identify which metrics are the most meaningful in the archival context; if we allow libraries to continue to apply uniform metrics across libraries and archives, the evidence of our impact is lost (connections to be explored here: the role of the professional organization/roundtables, schol comm - how to share this data, training and research - how to measure what needs to be measured, the integration of assessment and user experience into a primary service within larger university libraries)
Communicate and engage
Not simply how to engage, but more importantly, how to sustain engagement
This is where we discuss what we considered to be the major challenges in keeping the hackfest teams informed and engaged as the proposal was written
Ask what’s next - embed continuous learning and reflection into ecosystem
This is the overarching message that have evolved since into several different manifestations of the same message - here we can point to several of the sessions that seem like the continuation, the refinement of the cruder buckets that represented attempts to think iteratively, to embrace an ethos of experimentation: Agile in the Archives, Doing Digital Archives in Public, etc.
Research & development
R&D: Yan, Erjia. (2015.) "Disciplinary Knowledge Production and Diffusion in Science."article provides some evidence for the idea that R&D investment leads to knowledge production. Is “knowledge production” what we’re after in archives? Addressing the specific relevance of this question to digital archives and preservation. Is archives a knowledge/scholarship/research field or a service/product/system field

10. Conclusion/Article Coming Soon
Key thesis:
In order to tackle the challenges of curating and preserving emerging born-digital material at scale, archivists need to reframe research:
the way we approach it and where we conduct it
Digital archiving demands practitioner involvement in research and development; we need sustainable, participatory, translational research models in order to progress and succeed. Our project outlines a case study that can be extrapolated into one new model of research in practice.
[Jessica]
No landscape for born-digital access, did survey, then utilized further participatory data methods to create engagement/solutions building--twofold work.
In order to tackle the challenges of curating and preserving emerging born-digital material at scale, archivists and digital curators need to reframe research: the way we approach it and the places in which we conduct it (not just academia). Digital archiving demands practitioner involvement in research and development; we need sustainable, participatory, translational research models in order to progress and succeed. Our project outlines a case study that can be extrapolated into one new model of research in practice.