1. Knowing What We Learn and Demonstrating Our Success: Assessment and Evaluation of FLCs Andrea L. Beach, Ph.D. Western Michigan University Presented at the 2 nd Annual Faculty Learning Communities Conference Oxford, OH June 19, 2004

2. Purpose of this Session Overview of assessment and evaluation of FLCs, including authentic assessment Discuss results from Miami’s 3-year FIPSE funded FLC dissemination project

3. About the FIPSE Project... Miami University and six adapting institutions: Claremont Graduate University, Indiana University Purdue University Indianapolis, Kent State University, Notre Dame University, Ohio University, and The Ohio State University Project sought to test the “Miami Model” and the success of mentored, accelerated adaptation – 12 FLCs at each in 3 years.

4. Audiences and Reasons for Assessment and Evaluation of FLCs Participants and facilitators – for improvement of FLC processes Potential participants and facilitators – as recruitment tool Program director/FD director – for improvement of FLC program Institutional stakeholders whose support (financial and professional) is needed – chairs, deans, provost’s office – demonstrating outcomes and program value Larger academic community – this conference, NDTL, etc.

5. Evaluation Defined Evaluation – the systematic investigation of the effectiveness of a program (The Program Evaluation Standards, 1994). Formative and summative Measures pre-set goals and objectives, often institution, grant, or program-driven For FIPSE institutions, evaluation focused on effectiveness of institutions at adaptation, and effectiveness of FLCs for faculty development and student learning changes

6. Evaluation of FIPSE FLC Project 4 overarching evaluation questions: the impact on student learning faculty development outcomes the effectiveness of the program components of each FLC effectiveness and success of the project dissemination process in establishing a FLCP at each AI Data includes participant surveys, case reports, year-end reports, site visits, and discussions

7. Assessment Defined Assessment -- systematic gathering of information about component parts of the program/activity being evaluated FLCs are ultimately aimed at improving student learning through effective teaching, therefore, assessing the learning of FLC participants, changes in their teaching practice, changes in student learning, and the collective learning of FLCs forms the heart of the evaluation of FLC programs.

8. Authentic Assessment of FLCs Authentic assessment is designed to address learning that is meaningful to the learner and the skills and abilities needed to perform actual or real world tasks Approaches could include documented demonstrations, interviews, essays, case studies, and other writing samples, observations, and the construction of portfolios

9. Principles of Authentic Assessment for FLCs Authentic assessment requires faculty members to be actively engaged in the process (based on metacognition). Authentic assessment should take place in variety of carefully planned ways before, during, on completion of, and following the FLC Authentic assessment of FLCs is an individual and social contextual process – balancing individual and collaborative approaches

10. Assessment within the Miami FIPSE project FLC programs chose topics that best fit institution needs. Individual FLCs collaboratively planned their goals, activities, and assessment to a large extent. Individual projects & products, participant surveys with questions specific to FLC Collective reflection for case reports, input on final year-end report Program directors individually and collectively reflected on results of FLC assessment.

11. Learning Within and Across FIPSE FLCs From FLC assessments, keys to successful participation include: Multidisciplinary make-up Bonding experiences Projects to focus efforts (individual or collective) Trained/experienced facilitators Visible support (money, release time, recognition) Opportunities to gather and disseminate

12. Within and Across FIPSE FLC Programs Knowing where the right people and the money are can help get things rolling Partner with other departments (e.g., library, IT, Student Affairs) Plan recruitment that aligns with FLC goals Build a cadre of TRAINED facilitators Support can be offered in many forms – not just $$ -- be creative to work within your means

13. FLC Institutional Considerations Understand and work with unique institutional culture & current climate Address issues of institutional concern with FLCs Foster and maintain leadership buy-in through constant communication Show your FLCs off around campus (workshops, conferences, newsletters, etc.) Share your assessment and evaluation results

14. Other Observations Perfection is not the point – building an effective FLC program takes time and adjustment Success breeds imitation – expect and respect this! many FLC “like” groups may spring up in response to your success. This is positive, but keep them distinct from your FLC program. Not every group is an FLC. Know your limits – how many FLCs you can sustain depends on all of the factors discussed here. Don’t burn yourself out or risk “FLC fatigue” in the institution.

15. Conclusion Assessment and evaluation of FLCs are crucial to their long-term sustainability and success. Plan your assessment and evaluation approaches as you plan your FLCs and FLC programs. There are numerous resources available to help you – you don’t have to re-invent the wheel. Find assessment and evaluation instruments in the FLC Facilitator’s Handbook My website: http://homepages.wmich.edu/~abeach/

16. Questions?