1. Using the Capstone Course to Generate Student Learning Outcomes Texas State University Thomas E. Castleberry Patricia M. Shields Hassan Tajalli

2. Student Learning Outcomes A description of the knowledge, skills, values and abilities attained by students as a result of their educational experiences. Examples: Students will display effective written communication skills. Students will develop adequate knowledge of budgeting and financial processes.

3. Student Learning Outcomes describe specific behaviors that a student of your program should demonstrate after completing the program focus on the intended abilities, knowledge, values, and attitudes of the student after completion of the program –What is expected from a graduate of the program? –What is expected as the student progresses through the program? –three questions What does the student know? (cognitive) What can the student do? (psychomotor) What does the student care about? (affective)

4. Mission-Driven Assessment Outcomes = Educational Values = Mission Mission-driven assessment derives desired outcomes from the mission, and chooses outcomes that are designed to further that mission. Outcome measures then determine how successfully the mission is being accomplished.

5. Methods of Assessment Operationalize outcomes so that they can be measured, by providing evidence of their demonstration or existence. Examples: Tests, Papers, Surveys, Portfolio Evaluation, etc..

6. Measuring Learning Outcomes should provide an objective means of quantifying the outcomes, quality, efficiency or productivity of programs, operations, activities or services should indicate how you will measure each of your outcomes should indicate when you will measure each outcome provide at least two ways to measure each outcome

7. Direct and Indirect Methods Direct Methods: Processes deliberately designed to measure evidence of student learning by evaluating performance. e.g., Tests, Projects, Course Grades Indirect Methods: Processes that measure perceived student learning. e.g., Surveys, Transcript Analysis, GPAs

8. Describing Measurement Methods What are you going to use? –presentation, assignment, test, survey, observation, performance rating Of and/or by whom? –student, mentor, focus group, alumni Context (e.g., where or when)? –point-of-service, capstone, throughout the year, end of program For what purpose? –desired learning outcome example: Test the students at the end of the program for their level of knowledge in XYZ

9. Short, Intermediate, and Long-term Outcomes Outcomes for students studying to be practitioners should relate to job performance. Challenge: Short/Intermediate outcomes must accurately gage long-term career outcomes within the academic environment. Example Intermediate Outcome: Knowledge of effective management practices Long-Term Outcome: Demonstrates effective management practices

10. Capstones as Assessment Methods Provide evidence of the cumulative effect programs have on student learning. Less obtrusive and more comprehensive than stand-alone examinations. Students use higher-order thinking skills to combine prior learning experiences. Opportunities for multiple measures and involvement of external constituencies.

11. Bloom’s Taxonomy of the Cognitive Domain Knowledge: requires and answer that demonstrates simple recall of facts Comprehension: requires an answer that demonstrates an understanding of the information Application: requires an answer that demonstrates an ability to use information, concepts and theories in new situations Analysis: requires an answer that demonstrates an ability to see patterns and classify information, concepts and theories into component parts Synthesis: requires an answer that demonstrates the ability to relate knowledge from several areas to create new or original work Evaluation: requires an answer that demonstrates an ability to judge evidence based on reasoned argument.

12. Capstone Models Applied Research Project Group Project Internship Capstone Course

13. Texas State University MPA Program Mission The MPA Program's mission is to prepare students for careers as managers and leaders in the public service.

14. Texas State University MPA Program Learning Outcomes Students should demonstrate the ability to communicate effectively in writing. Students should demonstrate the ability to communicate effectively in orally. Students should demonstrate the ability to see patterns and classify information, concepts and theories in public policy and administration. Students should demonstrate the ability to judge public policy and management evidence based on reasoned arguments. Students should demonstrate knowledge and comprehension of National Association of Schools of Public Affairs and Administration’s accrediting standards curriculum components.

15. Texas State University MPA Program Student Learning Outcomes: Direct Measures Intended Student Learning Outcomes Assessment Mechanism Operationalization Strongly Agree//Agree//Neutral//Disagree// Strongly Disagree 1. Students will display effective written communication skills. ARP & Oral Exam: Assessment by outside practitioner 1. The student ’ s writing was clear 2. The student ’ s writing used correct grammar and punctuation. 3. The student ’ s use of referencing was appropriate. 2. Students will display effective oral communication skills. ARP & Oral Exam: Assessment by outside practitioner 4. The student ’ s speaking style was clear. 5. The student ’ s discussion communicated a mastery of the subject matter. 3. Student will be able to see patterns and classify information, concepts and theories in public policy and administration. ARP & Oral Exam: Assessment by outside practitioner 6. The student demonstrated the ability to see patterns and classify information, concepts and theories in public policy and administration. 4. Student will be able to judge public policy and management evidence based on reasoned arguments. ARP & Oral Exam: Assessment by outside practitioner 7. The student demonstrated the ability to judge public policy and management evidence based on reasoned arguments.

16. Texas State University MPA Program Student Learning Outcomes 5. NASPAA Curriculum Components Assess. Mechanism Operationalization of Knowledge and Skills To what extent did the ARP deal with the following topics. Somewhat Significantly Not At All  Management of public service organizations  Application of quantitative and qualitative techniques of analysis  Understanding of the public policy and organizational environment. Applied Research Project: Assessment by outside practitioners 1. Human Resource management 2. Budgeting or financial processes 3. Information management/technology applications 4. Policy and program formation 5. Program implementation/evaluation 6. Decision-making 7. Problem solving 8. Political/legal institutions & processes 9. Economic/social institutions & processes 10. Organization/management concepts 11. Ethical dilemmas or application

17. Texas State University MPA Program Student Learning Outcomes: Indirect Measures Intended Student Learning Outcomes Assessment Mechanism Operationalization Strongly Agree//Agree//Neutral//Disagree// Strongly Disagree 1. Students will display effective written communication skills. Student Exit Survey Collected at Oral Exam. The MPA contributed to the development of my skills in writing. 2. Students will display effective oral communication skills. Student Exit Survey Collected at Oral Exam. The MPA contributed to the development of my skills: a. Oral Communication b. Presentations 3. Student will be able to see patterns and classify information, concepts and theories in public policy and administration. 4. Student will be able to judge public policy and management evidence based on reasoned arguments.

18. Texas State University MPA Program Student Learning Outcomes 5. NASPAA Curriculum Components Assess. Mechanism Operationalization of Knowledge and Skills To what extent did the ARP deal with the following topics. Somewhat Significantly Not At All  Management of public service organizations  Application of quantitative and qualitative techniques of analysis  Understanding of the public policy and organizational environment. Student Exit Survey Collected after Oral Exam The MPA program contributed to my knowledge of a.Decision making/problem solving b.Financial resources c.Human resources d.Policy process e.Ethics f.Information systems technology g.Law h.Public Management i.Organizational processes

19. Texas State University: MPA Program Student Learning Outcomes Direct Method: Practitioner Evaluation Writing Ability95% Positive Oral Communication93% Positive Analytical Ability82% Positive Reasoning85% Positive Human Resources33% Significant Coverage Budgeting/finance30% Significant Coverage Info. Mgm/Tech Application31% Significant Coverage Policy &Prog. Formulation70% Significant Coverage Program Imp./Eval.85% Significant Coverage Decision-making63% Significant Coverage Problem solving67% Significant Coverage Political/legal Inst. & Processes44% Significant Coverage Eco./Sco. Inst. & Processes48% Significant Coverage Org./Mgmt concepts56% Significant Coverage Ethical Dilemmas or Application33% Significant Coverage

20. Texas State University: MPA Program Student Learning Outcomes Indirect Method: Student Exit Survey KNOWLEDGE AREA Decision Making Problem Solving97% Positive Financial Resources89% Positive Human Resources95% Positive Policy Process95% Positive Intergov. Relations90% Positive Ethics97% Positive Info. System Technology81% Positive Public Management94% Positive Organizational Theories96% Positive Applied Research99% Positive Law79% Positive SKILLS Writing98% Positive Quantitative Analysis97% Positive Oral Communication95% Positive Team work83% Positive Financial Analysis87% Positive Presentation93% Positive Interpersonal Relationships86% Positive Project Management89% Positive Supervisory79% Positive Program Evaluation83% Positive Assessment92% Positive Use of Technology80% Positive

21. Re-Cap of Process Step 1: Define Program Mission Step 2: Define Program Goals Step 3: Define Student Learning Outcomes Step 4: Identify assessment methods and targets for each learning outcome Step 5: Operationalize the Assessment Method Step 6: Findings