Faculty use of digital resources and its impact on digital libraries Josh Morrill, Morrill Solutions Research Alan Wolf, University of Wisconsin - Madison Ellen Iverson, Carleton College Flora McMartin, Broad Based Knowledge Cathy Manduca, Carleton College Glenda Morgan, California State University

Outline Background Focus groups Survey Design Survey delivery Current results Follow up

Study background Research questions What are the characteristics of online collections that make them useful for teaching? How do faculty employ materials in useful collections? How are collections, resources, and services best aligned with faculty work patterns?

Overview of methodology Focus groups Code transcripts & determine themes Survey design Institution recruitment Survey delivery Analysis

Focus groups Conducted across the range of higher education institutions, STEM disciplines, and instructor ranks. Questions Finding materials (types, methods, barriers) Sharing materials (types, methods, barriers) Communities and their roles in teaching improvement and sharing. How they use the web for professional development

What they know about DLs Personal definitions of DLs vary widely Very few people knew about NSF DL efforts Barriers Information overload Concern about copyright and use Not invented here Google as content aggregator Concern about the availability of resources

Themes arising from the focus groups Characteristics of online collections that make them useful Faculty work patterns vis a vis digital learning materials and teaching Alignment between DLs and faculty work patterns Faculty use of DLs and digital materials Obstacles to faculty use of DLs Faculty use of Web for P&T purposes Bullet 1 is “The kinds of content, features/services that attract users to the collection and that they return to because of these characteristics” Bullet 2 is “This theme attempts to identify the work patterns of faculty members as they prepare for a class or course. The section spans from the planning process through teaching” Bullet 3 is “What are the features and services that faculty desire and which ones do digital libraries provide? How do faculty perceive these services and features? What might make them easier to use?” Bullet 4 is “Level to which faculty use digital libraries, if at all and why they say they do or don’t. What gets in the way of their use of these services? “ Bullets 5&6 are self explanatory

Survey design Three Facets of Survey Validity Face Validity External Validity Internal Validity

Face Validity (Def) Does this “seem” right? Does this survey address what we want it to address? How we addressed Face Validity: Developed survey over 5 months (Feb – Jun) with grant research group. Open process with extensive feedback, meetings and revisions.

External Validity (Def) Do other people think this survey seems “right”/ cohesive/ clear? How we addressed External Validity: Pre-Tested survey on appox. 20 faculty members (R1, technical college, masters and liberal arts colleges) 6 were followed up with in depth interviews. The remainder were given a survey with comment boxes on each page.

Internal Validity (Def) Are you measuring what you THINK you are measuring? How can you minimize error in your measurement? How we addressed Internal Validity: “Reverse Coded” items to assess mindfulness. Likert Scales Throughout: forced choice vs. ambivalence scale points. (Skip Logic helps this immeasurably) Factor Analysis/ Reliability Testing (more to come here)

Recruiting Recruiting is ongoing We have attempted contact with all higher education institutions for whom we could identify contacts. Contacts attempted were typically library directors, deans of science colleges, learning technology centers, and occasionally chairs of science departments

Recruiting We have have received responses from almost 150 institutions More than 75 have agreed to participate The majority have sent the survey to all their faculty (all disciplines, all ranks). There has been an outstanding response from the community colleges.

Recruiting - what is left to do We believe that given the current rate of participation we are on track to gather statistically significant sample We may need to recruit more doctoral and master’s degree granting institutions

Preliminary results To date, much of the focus group data is being confirmed --- But data is still coming in. ….AND WE STILL WANT MORE! But the level of granularity in the survey is giving some unique nuance/ differentiation to the focus group data. For example…?

How does this compare with the focus groups? Low-complexity High granularity Google Survey Low-complexity High granularity Google BUT… There are some interesting trends emerging

Follow up - Faculty development The original idea for this grant Analyze which practices work under which circumstances Provide guidance for faculty developers as to which practices might work best for local circumstances.

Follow up - web search observational studies Survey gives ½ the story --- But what happens BEFORE people enter a digital library? What are the decision points in a search process that may now lead to “Google” but could be changed to lead to a digital library?

Contact Information If you are interested in participating Contact alanwolf@wisc.edu For more details visit http://serc.carleton.edu/facultypart We wish to acknowledge the National Science Foundation for their support (DUE-0435398).