Evaluation and Research: Dual Purpose Data Collection Ann E. Austin Professor, Michigan State University ADVANCE PI Meeting March 5, 2013

Key Questions addressed in This Session How to use evaluation data in research How to frame single institution data for publication

Suggestion 1: Take a dual purpose perspective from the start, rather than waiting until late in the project or after it is finished – –Take the time to assemble your project team, including internal and external evaluators, to discuss project goals, and ways to engage in both evaluation and research

Suggestion 2: Consider the differences between evaluation and research questions – –Framing questions from the point of view of advancing knowledge (research) versus asking questions inform and assess project development, outcomes, and institutional practice (evaluation)

Suggestion 3: Read literature that enables one to see connections, move from local examples to broader questions, and situate evaluation questions within larger contexts – –Consider the literature in sociology, educational psychology, political science, higher education, and women’s studies related to organizational change, leadership, faculty work and roles, gender issues, etc.

Suggestion 4: Use a conceptual framework to view issues and questions (which helps to situate what one is studying in a context beyond a single case) – –Conceptual frameworks highlight key questions, relationships, factors, and variables to consider in your work – –Conceptual frameworks help situate local questions or issues in ways that may be relevant to other institutions and to scholars studying related issues

Suggestion 5: Take note of emerging questions and observations that may lead to broader research – –As the project develops, observe and note the issues that emerge, including challenges, barriers, and facilitators. Consider how your observations either confirm or contradict what you have seen in the related literature. These may be areas for shifting evaluation questions toward research questions.

Suggestion 6: Ensure that IRB approvals are in place from the start so that data can be used for publication – –All data used in research publications must have been collected under appropriate IRB guidelines and approvals. – –A good practice is to have clear lines of communication with the IRB office and ensure that all data collection processes have been approved.

Suggestion 7: Take the time to step back and ask what evaluation results mean or imply, and whether others may find results useful – –Consider allocating some project team meeting to reviewing evaluation results, discussing their meaning, interpreting them in light of broader literature, and considering if the emerging findings may be useful beyond your institution.

Example: Evaluation Leading to Research Internal and external evaluation work at an ADVANCE institution led to interesting questions about the relationships between organizational characteristics and culture to interventions and strategies used and their impact These questions led to an ADVANCE IT project: A. E. Austin & S. Laursen’s study concerning: –Strategies and processes that have worked well in creating institutional environments conducive to the success of women scholars –Why and under what circumstance these interventions worked –How institutional culture and characteristics relate to choice and effectiveness of strategies

Contact Information Ann E. Austin Professor, Higher, Adult, and Lifelong Education 419A Erickson Hall 620 Farm Lane Michigan State University East Lansing, MI Tel: