Robert B. Sweeney, Jr. Kelly Mosteller, Masters Graduate Student Roy J. Daigle University of South Alabama School of Computer and Information Sciences Mobile, Alabama (334) An Examination of the Relationship Between Active Participation in Test Development (APTD), Student Performance and Student Attitudes

BACKGROUND Goal: incorporate higher levels of skills/knowledge into computing curriculum

Background: Tests Useful for Assessment Instruction Retest Collaboration Feedback

Background: Test Issues Poor preparation Uncertainty Project Generalization

Background: APTD Active Participation in Test Development (Daigle & Doran, 1998) Designed for Project Generalization objectives Students & instructor develop test in collaboration Four phases: 1. Call for test item submission 2. Cooperation for integration & review to prepare test 3. Instructor review of prepared test 4. Instructor preparation of the examination. Encourages early student exam preparation, reduces student uncertainty, and provides feedback to instructor

RESEARCH OBJECTIVES Determine if there is a relationship between the use of APTD and student performance on examinations Predictor Variables: Quantity and Complexity of questions submitted Depth of Knowledge scale based on Bloom Taxonomy used to assess Complexity Assess student opinion relative to course subject matter and use of APTD

HYPOTHESES Relationship between APTD and Student Exam Performance H1: The quantity of student questions submitted as part of APTD will not affect the students test performance. H2: The complexity of student questions submitted as part of APTD will not affect the students test performance. H3: The interaction of quantity and complexity of the student questions submitted as part of APTD will not affect the students test performance. Student Opinion A student survey was conducted to determine their preferences for particular class subject material and for their opinion of APTD

METHODOLOGY Subjects: students in accelerated database- programming course preparing for Computer Science or Information Systems masters Prerequisites: accelerated courses in Java, data/file structures, architecture/operating systems, and networks/communications

RESULTS – H1, H2, and H3 Correlation analysis performed comparing midterm and final exam scores with: H1: Quantity of questions submitted No significant correlation coefficients H2: Complexity of questions submitted No significant correlation coefficients H3: Quantity and Complexity of questions submitted No significant correlation coefficients Consequently we cannot reject H1, H2, or H3

RESULTS – SURVEY Students reported generally high interest in all subject matter categories Practical, project-oriented categories (MS Access and Visual Basic, and relational DBMS) rated higher than theoretical (normalization) Students reported generally strong agreement with the effectiveness of APTD Relatively lower support for idea that APTD helped in student exam preparation or exam performance

DISCUSSION Lack of correlation between exam performance and APTD might be due to: Rewording of submitted exam questions to a higher complexity level Lack of specific instruction on levels of knowledge complexity Frequency of question submissions Student perceptions were that the APTD approach was effective.

FUTURE DIRECTIONS Revise methodology and redo experiment More depth of knowledge instruction Adjust number of requested submissions Larger/more diverse subject group