Assessment in the Biology Department in AY Caroline Solomon Friday December 5

Biology Department Outcomes The development of the biology department outcomes took several years (thanks to Dr. Ann Powell who led us in this effort) before finally being polished by Bette Landish

Biology Department Outcomes 1. Demonstrate knowledge of current theories of biology and concepts basic to biology, including: Levels of complexity (molecular/cellular through population/communities/ecosystems); biological principles and processes; connections among concepts across disciplines (physical sciences, mathematics, social sciences). 2. Demonstrate critical thinking skills through: Identifying appropriate questions, issues or problems to be explored; analyzing, interpreting and evaluating evidence; constructing well-supported, clearly articulated arguments and conclusions.

Biology department outcomes (con’t) 3. Demonstrate information literacy and research skills through effective use of: Sources of information in biology including published literature and scientific databases; computer applications to acquire and (continued) analyze experimental data and develop graphic models. 4. Demonstrate experimental research methods including: Scientific methods and instrumentation; safe and appropriate use of laboratory equipment; experimental design; data analysis; familiarity with professional standards in science.

Biology department outcomes (con’t) 5. Demonstrate effective communication with varied audiences and media by: Providing clear structure and transitions; demonstrating substantial understanding via research, credible sources and supporting evidence; demonstrating audience-appropriate purpose, agenda, language and style. 6. Demonstrate preparation for future career and educational goals by: awareness of personal competencies (strengths and weaknesses); ability to work in a team to accomplish a goal; understanding professional and ethical behavior.

Four year plan for assessment OutcomeAY AY AY AY #1: Biology Content ETS Field Test; Senior Exit Surveys #2: Critical Thinking Development of assessment tools #3 : Information literacy Begin assessment #4 Experimental research methods Begin assessment #5 Effective communication Writing & Presentation rubrics #6 Career Preparation Internships portfolios

Demonstrate knowledge of current theories of biology and concepts basic to biology, including: Levels of complexity (molecular/cellular through population/communities/ecosystems); biological principles and processes; connections among concepts across disciplines (physical sciences, mathematics, social sciences). (1) Direct measurement: ETS Biology Field Test (2) Indirect measurements: (a) Revised student evaluations to reflect outcomes (b) Exit surveys from seniors (2007 & 2008) Outcome #1: Biology content

Results from ETS Biology Field Test *areas that need to be worked on the most

Student distribution *Subscores are reported; (% compared to institutional mean scores that includes seniors from 345 domestic institutions)

Our next step… closing the loop for Outcome #1 Weaknesses: biochemistry, cell biology & botany/plant physiology How do we change our curriculum or courses to address these weaknesses? Department decided to: (1) emphasize botany and plant physiology more in second semester of introductory biology (BIO 108) and in ecology (BIO 241) (2) strongly encourage students to take cell biology or molecular biology; offer the course as early as Spring 2009

Outcome #5: Effective Communication Demonstrate effective communication with varied audiences and media by: Providing clear structure and transitions; demonstrating substantial understanding via research, credible sources and supporting evidence; demonstrating audience-appropriate purpose, agenda, language and style. (1) Direct measurement: Rubrics for lab reports, literature reviews & presentations (2) Indirect measurement: Exit surveys from seniors (2007 & 2008)

Assessment tools for this outcome (1)Writing a. Lab report rubric b. Literature review rubric (2)Presentations a. Class presentation rubric

Lab report rubrics Two different lab rubrics were used during AY Why did we revise the lab report rubric for Spring 2008? – A lab report rubric workshop was held in December 2007; the scoring for three different student products were not consistent among department members

Lab report rubric used in Fall 2007

Revised lab report rubric

Lab Reports – Fall 2007 BIO 103; n= 48 BIO 108; n= 32 BIO 231/241; n = 17 We identified these problem areas in December 2007; Intro & discussion involve critical thinking

Lab Reports – Spring 2008 BIO 104/GSR 150; n= 85 BIO 110; n= 33 BIO 211,221, Capstone; n =30 Note difference in scales from Fall 2007 (5 points) to Spring 2008 (4 points)

A closer look…..Title and Abstract Need to figure out why With the new lab report rubric – we can now see things in more detail

A closer look….Introduction Decline because SQ is given in earlier courses?

A closer look….Materials & Methods Senior Exit survey: Did the program teach you how develop your own experiments? 2007 – 3.7; 2008 – 2.5 Did the program prepare you to use various laboratory instruments? 2007 – 4.2; 2008 – 3.6 Students need more work in developing their OWN experiments

A closer look….Results Senior Exit survey: Did the program teach you how to analyze, interpret and evaluate evidence? 2007 – 4.7; Did the program teach you how to develop good graphs and charts? 2007 – 4.2; 2008 – 2.4 Students need more practice interpreting & creating their own graphs

A closer look….Discussion & Conclusion Senior Exit survey: Did the program teach you how to support your arguments and conclusions? 2007 – 4.4; 2008 – 2.8 Still needs improve- ment

A closer look….Quality of Writing

A closer look….Citation & Bibliography Improvements were made in this category from fall to spring

Synthesis of results of writing assignments Decrease in ability from freshman year to upper level courses: – To write a good abstract – Identification of scientific question Areas that need improvement – Development of own experimental procedures (identification of controls) – Data analysis; graphing – Discussion & Conclusion (critical thinking)

Our next step… closing the loop for Outcome #5 The discussion of the results from the lab report rubric produced a healthy discussion about comfort levels of different faculty members about Excel (data analysis & graphing) and defining levels of expectation of mastery in Excel were needed

Presentations Assessment of presentations during Fall 2007 were compounded by the use of different rubrics by different professors, no recording of presentations In January 2008, the department developed a presentation rubric that was used in Spring 2008

Presentations – Fall 2007 Note: For most categories n= 20, but some had n= 26 due to different rubrics; for better understanding of categories see journal club rubric (BIO 231, 241, 401)

Presentations – Spring 2008 n=3 n=8

Improvements that needed to be made in collection of data for presentations Identification of weaknesses: – Content – Use of visual media – Bilingual communication (sign clarity) Often people did not do all the categories No recording of presentations; collection of evidence – A new protocol to record presentations for Fall 2008

Focus for AY

Four year plan for assessment OutcomeAY AY AY AY #1: Biology Content ETS Field Test; Senior Exit Surveys #2: Critical Thinking Development of assessment tools #3 : Information literacy Begin assessment #4 Experimental research methods Begin assessment #5 Effective communication Writing & Presentation rubrics #6 Career Preparation Internships portfolios

Outcome # 2: Critical Thinking Demonstrate critical thinking skills through: Identifying appropriate questions, issues or problems to be explored; analyzing, interpreting and evaluating evidence; constructing well-supported, clearly articulated arguments and conclusions. Suggestions for measurements: (1) Critical thinking components of writing & presentation rubrics (2) Inclusion of data analysis problems and conceptual problems in exams in major courses (following Kitchen et al. 2003) (3) Scoring on analytical problems on the ETS Field Test

Focus for Spring 2009 Invite Virginia Anderson from Towson to present on how to assess critical thinking in biology (coming January 15 th ) Develop assessment tools for critical thinking (outside of writing rubrics)

Outcome #6: Career Preparation (Ava) Demonstrate preparation for future career and educational goals by: awareness of personal competencies (strengths and weaknesses); ability to work in a team to accomplish a goal; understanding professional and ethical behavior. (1) Direct measurement: Alumni surveys (how many are employed in the field after graduation?) (2) Indirect measurements: (a) Internship portfolio (b) Exit surveys from seniors (2007, 2008 & 2009)

Reflections on assessment At the first department meeting of the AY, we had three groups of data to help us guide us in making decisions about the future direction of the department: (1) Assessment data from the 2 outcomes (2) Preliminary data from internship portfolios (3) An analysis of Biology curriculums around the country at schools of a comparable size to Gallaudet

Even if we are starting to develop tools for the two student outcomes we want to measure in AY , we are continuing to collect data on the two learning outcomes we started last year - it’s an ongoing process!