1 Identifying Instruction-Related Research Issues Deborah Lines Andersen School of Information Science and Policy University at Albany June 26, 2004

2 A Research Story The research problem Methodology decisions Respondent choices The “so what?” Data collection Data analysis Results in action

3 Transformation Turning an idea into a workable research process Using standard research methods Relying upon others for feedback and guidance (and creating “buy in”) Piloting before conducting research Assessing the costs and benefits up front

4 An Idea We want to find out if we are offering the right courses in the library. Should we be offering different courses? Which ones? How often and when? Should we use budget dollars to develop new curricula?

5 Asking the Right Questions Of ourselves Of our staff Of our patrons Of our administrators Of our collections and services Of our educational materials Of other units in our organization

6 Asking the Right Questions How will this research be used? –Formative evaluation? –Summative evaluation? –Change internal to the organization? –Dissemination external to the organization? –Allocation of existing resources? –Justification for new resources?

7 What Do We Want to Assess?

8 What Do You Want to Assess? Create a one-sentence statement of the problem that you are addressing Create a second sentence describing why this is an important problem Write a third sentence explaining how doing this research will help your organization, your unit, and/or your users

9 An Idea (Again) We want to find out if we are offering the right courses in the library. Should we be offering different courses? Which ones? How often and when? Should we use budget dollars to develop new curricula?

10 The Transformation The problem: The university library needs a systematic review of it course offerings (low attendance, poor post-class evaluations). The importance: With finite staff and funding the library wants to offer the best courses with it’s existing resources. The benefit(s): Appropriate courses will attract more students and faculty, making more patrons better users of the library’s services.

11 Who Has the Information You Need for Assessment?

12 Who Has the Information You Need for Assessment? Find the right respondent pool Worry about generalizability and statistical significance (qualitative vs quantitative) Look at (some or all?) –Learners/patrons –Teachers/librarians –Organizations/administrators

13 How Will You Conduct This Research? (Methodology)

14 How Will You Conduct This Research? (Methodology) Assess the best way (time, labor, and cost) to collect the necessary data Consider the “usual” methods: –surveys –interviews –focus groups –secondary data analysis

15 How Will You Conduct This Research? (Methodology) Consider partnerships –Other librarians (internal and external) –Seminars to discuss research old and new –Graduate students as staff –PhD students as researchers –Faculty and other staff (IT, research agencies) –Schools of library and information science

16 A Methodology Statement In order to assess the appropriateness of library course holdings, a paper survey will be developed, piloted, administered to 400 undergraduates in their required English classes, and analyzed between September 2004 and April 2005 to determine the effectiveness of present classes and the need for new and/or revised course offerings.

17 What Questions Should You Ask?

18 What Questions Should You Ask? (Think “Parsimony”) Use existing data collection documents Pilot test new materials Think about analysis issues (charts, stats) Check for bias, “doubled-up” questions Think about length of individual data collection instrument (response rates) Check logical order of questions

19 Does This Research Meet Organizational Standards?

20 Does This Research Meet Organizational Standards? Research review boards and procedures Human subjects review Research ethics Research involving minors Confidentiality and anonymity Access to and disposal of data Classes on human subjects and ethics

21 How Long Will It Take to Do the Research?

22 How Long Will It Take to Do the Research? Creating committees Creating methodological protocols Securing organizational approval Securing funding (internal or external) Contacting respondents Waiting for respondents Coding, analyzing, and reporting findings

23 A Technology Note on Data Analysis Quantitative Analysis –By hand –Excel –SPSS … Qualitative Analysis –By hand –Atlas.ti; Nudist; …

24 How Much Will It Cost? Who Will Pay for This? (Grants???)

25 How Much Will It Cost? Who Will Pay for This? (Grants???) Budget for research Assess direct (and indirect) costs Use existing staff and funds Seek grants inside or outside your organization Use existing data Pay attention to expensive data

26 Research Project Action Plans: Practical Next Steps 1 Identify your group’s facilitator Assign a time keeper (30 minutes plus 15 minute break) Do a round-robin reading of the group’s problem statements Ask questions about each other’s projects Rewrite/edit problem statements; then write importance and benefit statements

27 Research Project Action Plans: Practical Next Steps 2 Identify appropriate subjects and stakeholders Identify appropriate methodology or methodologies Consider the best time frame for the research List possible publication venues Create your action plan

28 Get Started!