CBU Online Survey System MASEC March 31, 2006 John Ventura

Teacher Evaluations

Project Description Current methods at CBU Current methods at CBU Paper survey Paper survey Count results by hand or scantron Count results by hand or scantron I.T.S. worker creates a web page I.T.S. worker creates a web page Takes a lot of time and effort Takes a lot of time and effort

Project Description Easy to use web-based evaluation tools: Easy to use web-based evaluation tools: Create, edit and administer surveys from a web site Create, edit and administer surveys from a web site Export and import information to and from a personal computer Export and import information to and from a personal computer Manage users and groups, with usernames and passwords Manage users and groups, with usernames and passwords Analyze results with different types of table and graphs, and compare information for a survey over several terms. Analyze results with different types of table and graphs, and compare information for a survey over several terms.

ABET Accreditation Board for Engineering and Technology (ABET) Accreditation Board for Engineering and Technology (ABET) Web-based Evaluation Process Web-based Evaluation Process Data base for developing trends Data base for developing trends

Project Goals Analyze available products on the market. Analyze available products on the market. Design a software system to create and manipulate data in a survey. Design a software system to create and manipulate data in a survey. Implement software system suitable for the Christian Brothers University environment. Implement software system suitable for the Christian Brothers University environment.

Important Points Levels of user access Levels of user access Department Head and Administration Department Head and Administration Faculty and Staff Faculty and Staff Students Students Alumni and Guests Alumni and Guests

Important Points Security Security Access based on user login and password similar to CBU Web mail Access based on user login and password similar to CBU Web mail Import a text file with student usernames Import a text file with student usernames Create students one at a time Create students one at a time User identification used only for login and not to indicate which surveys have been taken User identification used only for login and not to indicate which surveys have been taken

Additional Functionality User login verification User login verification Import / export data Import / export data Adding, editing and deleting response sets Adding, editing and deleting response sets Help pages and tutorials Help pages and tutorials

Research Method (a) establish an ECE Curriculum Committee, (a) establish an ECE Curriculum Committee, (b) expand the responsibilities of the ECE Advisory Board, (c) develop Web-based instruments to measure the achievement of program objectives of graduates and learning outcomes of students, and (d) provide a model that integrates input from constituents, assessment committees, and results formulated from the results of measuring achievement of students and graduates into an evaluation process.

Procedures Employee surveys/questionnaires to measure the achievement of program objectives using Web-based technologies Employee surveys/questionnaires to measure the achievement of program objectives using Web-based technologies Establish an ECE Curriculum Committee that contains members from local engineering societies and organizations, the student chapter of IEEE, and faculty Establish an ECE Curriculum Committee that contains members from local engineering societies and organizations, the student chapter of IEEE, and faculty Expand the responsibilities of the ECE Advisory Board to include the examination and validation of the evaluation processes Expand the responsibilities of the ECE Advisory Board to include the examination and validation of the evaluation processes Provide constituents with instruments that measure the level of achievement of program objectives Provide constituents with instruments that measure the level of achievement of program objectives Provide results of surveys/questionnaires to the ECE Advisory Board and ECE Curriculum Committee that enabled them to validate the survey instruments and the model for evaluation Provide results of surveys/questionnaires to the ECE Advisory Board and ECE Curriculum Committee that enabled them to validate the survey instruments and the model for evaluation

ECE Curriculum Committee A member of the executive board of the Memphis Chapter of IEEE A member of the executive board of the Memphis Chapter of IEEE A member of the executive board of the Memphis Chapter of the Tennessee Society of Professional Engineers (TSPE) A member of the executive board of the Memphis Chapter of the Tennessee Society of Professional Engineers (TSPE) A member of the Organizing Committee of the Memphis Area Engineering and Sciences Conference A member of the Organizing Committee of the Memphis Area Engineering and Sciences Conference Chair of the IEEE Student Chapter at CBU Chair of the IEEE Student Chapter at CBU ECE faculty ECE faculty Dean of Engineering Dean of Engineering Chair of the Master of Engineering Management Chair of the Master of Engineering Management A faculty member from the School of Sciences A faculty member from the School of Sciences

1Strongly Disagre e Strongly Not Confident PoorNot at AllNoneNot Applicable 2DisagreeNot ConfidentFairVery Little Unimportant 3Neutral GoodSome Important 4AgreeConfidentVery goodA Lot Very Important 5Strongly Agree Strongly Confident ExcellentA Great Deal Essential Rating Scale

Grading Criteria for ABET Learning Outcomes for Industry Survey Grade Criteria Grade Criteria A+ >= 80% of responses in categories 3 and 4; > = 50% rated as 3 A+ >= 80% of responses in categories 3 and 4; > = 50% rated as 3 A >= 80% of responses in categories 3 and 4; > = 37.5% rated as 4 A >= 80% of responses in categories 3 and 4; > = 37.5% rated as 4 A– >= 80% of responses in categories 3 and 4; = 80% of responses in categories 3 and 4; < 37.5% rated as 4 B+ 60 to = 37.5% rated as 4 B+ 60 to = 37.5% rated as 4 B 60 to = 25% rated as 4 B 60 to = 25% rated as 4 B–60 to < 80% in categories 3 and 4; < 25% rated as 4 B–60 to < 80% in categories 3 and 4; < 25% rated as 4 C+ Highest frequency of ratings for category 2 and 3 but number of (1+2) C+ Highest frequency of ratings for category 2 and 3 but number of (1+2) C 60 to < 80% in category 2 and 3 C 60 to < 80% in category 2 and 3 C– Highest frequency of ratings for category 2 and 3 but number of (3+4) C– Highest frequency of ratings for category 2 and 3 but number of (3+4) D+ = 70% in categories 1 and 2; = 70% in categories 1 and 2; < 25% are in category 1 D = 70% in categories 1 and 2; >= 25% to = 70% in categories 1 and 2; >= 25% to < 37.5% are in category 1 D– = 70% in categories 1 and 2; >= 37.5% are in category 1 D– = 70% in categories 1 and 2; >= 37.5% are in category 1 F >= 90% are in categories 1 and 2 F >= 90% are in categories 1 and 2

ECE Advisory Board and ECE Curriculum Committee Validate instruments that contain the expectations of constituents, especially graduates and employers. Validate instruments that contain the expectations of constituents, especially graduates and employers. Assess the quality and performance of the evaluation process. Assess the quality and performance of the evaluation process. Assess the methods used for gathering information in a Web-based environment. Assess the methods used for gathering information in a Web-based environment.

Evaluation ECE Advisory Board evaluates the level of achievement of program objectives and evaluation processes ECE Advisory Board evaluates the level of achievement of program objectives and evaluation processes ECE Curriculum Committee evaluates survey instruments and evaluation processes ECE Curriculum Committee evaluates survey instruments and evaluation processes ECE Curriculum Committee evaluates the level of achievement of learning outcomes ECE Curriculum Committee evaluates the level of achievement of learning outcomes

Formative Evaluation Formative evaluation processes should be Formative evaluation processes should be undertaken due to the unfavorable factors inherent in information obtained from student self-assessment undertaken due to the unfavorable factors inherent in information obtained from student self-assessment Summative processes based on self-assessment is questionable Summative processes based on self-assessment is questionable Improvements in programs should be based on trends developed from data obtained over several semesters rather than a summative assessment of asingle cycle of an evaluation process Improvements in programs should be based on trends developed from data obtained over several semesters rather than a summative assessment of asingle cycle of an evaluation process

Online Survey System Demonstration

Any Questions?