1. Faculty Academy on Teaching First-Year students
Welcome
Wednesday, April 18th

2. How do you know your students are learning?
Outcome
> Identify classroom assessment techniques to assess CLOs
You can refine your teaching philosophy and advance your teaching by building a process of assessment and reflection to truly enhance the learning environment. This approach will provide you with evidence to define your teahing practices on how you engage, stimulate, and truly provide an effective learning environment.
The course evaluations tells you nothing about how students are learning, what they are learning, and the learning environment.

3. Course Learning outcomes (CLO)
What are CLOs?
Where are CLOs communicated to the students?
How often do you communicate the CLOs?
How do you assess CLOs? If so, which CATs?
What do you learn in this process?

4. Pre-post tests Self-Assessment: Public health course
Circle the number that best represents your skills before and after taking the class (5min)
Rating scale: 1=Low 3=Medium 5= High
Before Taking the class
Self-Assessment of skills development
After Taking the class 1 2 3 4 5
a. Explain and apply health communication theories to public health problems
b. Identify the advantages and disadvantages of the vast number of communication channels and demonstrate the ability to select the appropriate channel mix for any given public health communication objective and target audience
c. Formulate a plan for a focused communications strategy and provide a rationale, reflecting both scientific and practical considerations
d. Formulate logical, culturally appropriate, and measurable campaign objectives
e. Identify and apply appropriate research strategies for community and audience needs assessment, campaign and message design, and evaluation
f. Design communication messages and materials that respond to the needs of culturally diverse and low-literacy audiences
g. Articulate the ethical dilemmas inherent in the use of strategic communications to achieve public health objectives

5. Pre-Post Test Responses
Student Learning Outcomes (SLOs)
Rating Before Course
Rating After Course
Rating Increase 1 2 3 4 5 +0 +1 +2 +3 +4
A. Explain and apply health communication theories to public health problems 15 7 18 6 10 11
B. Identify the advantages and disadvantages of the vast number of communication channels and demonstrate the ability to select the appropriate channel mix for any given public health communication objective and target audience 8 12
C. Formulate a plan for a focused communications strategy and provide a rationale, reflecting both scientific and practical considerations 19 9
D. Formulate logical, culturally appropriate, and measurable campaign objectives 17
E. Identify and apply appropriate research strategies for community and audience needs assessment, campaign and message design, and evaluation 13
F. Design communication messages and materials that respond to the needs of culturally diverse and low-literacy audiences
G. Articulate the ethical dilemmas inherent in the use of strategic communications to achieve public health objectives 14

6. Follow up questions:
1) If you circled a high number,/change in rating from before and after/ what comments do you have to share? How? Why?
a)   What one or two assignments contributed the most to the achievement of this ability?
b)   Please briefly describe what made this assignment(s) so helpful? 
2) If you circled a low number, what comments do you have to share? How? Why?
a)   Do you have any recommendation for your instructor to further advance your ability to ...?
 3) For those who circled a medium number, what comments do you have to share? How? Why?

7. Follow up questions responses
SLO A: Explain and apply health communication theories to public health problems
What helped SLO A development: 1) the project in which students chose a public health issue was helpful in developing this skill (13/14 or 93%); 2) Lectures made the textbook material clearer (13/14 or 93%); 3) during class discussions, students were exposed to their peers’ perspectives, in addition to those presented by instructor or in the book (13/14 or 93%); 4) real life examples helped (11/14 or 79%); and 5) students agreed that having the instructor repeat ideas and explain how they related to new topics was helpful (9/14 or 65%).
Illustrative comments
“The instructor continual repetition of ideas, relating to new topics, and giving all the description was really helpful.”
“I learned more from lecture than from the book because in lecture we would speak as a whole group and get feedback from other students. It was really helpful to learn not just what the instructor thought about certain problems, but what students felt about public health problems.”
Follow up questions responses
Average Rating Before
Average Rating After
Average Rating Increase
1.71
4.07
2.43

8. Survey with Likert Scale
Self Assessment from Applied Math course
A) Please indicate the degree to which you agree with the following statement.
As a result of my preparation in the program, I am able to model real-world problems mathematically.
Strongly Agree
Somewhat Agree
Agree (neutral)
Somewhat Disagree
Strongly Disagree 
B) Briefly explain your response. Please give some concrete examples of practices that are helpful. Feel free to offer suggestions for improvement.

9. Survey with Likert Scale Results
Please indicate the degree to which you agree with the following statement:
As a result of my preparation in the program, I am able to model real-world problems mathematically.
About half of the students, 7 out of 15 (47%), selected that they agree/ neutral about feeling prepared to model real-world problems mathematically.
# of students*
Illustrative Comments*
Strongly Agree
Somewhat Agree 4
“So far my differential equations class, numerical analysis and linear algebra have taught me how some problems can be modeled mathematically”
Agree (neutral) 7
“I feel like I know what to do, yet I feel like I don’t understand how I can apply this to a future career”
Somewhat Disagree
“As a math major my future entails a life of doing mathematics through the use of computer, but I feel very unexposed to the topic”
Strongly Disagree

10. Posters around the classrooms
Poster could include:
Picture
CLOs (how well they have achieved them)
Readings for the class
Assignments
Activities
(which have led you to the achievement of the SLOs)
For the students to check or add.

11. Class Interviews with CLOs on board
Write the CLOs on the board and carry out a standard class interview. 1) What helps your learning in this class? 2) What suggestions would you provide to your instructor to help your learning? The instructor should … 3) What actions would you take to improve your learning? I should … Follow the think-pair-share approach.

12. How do you know your students are learning?
Outcome
> Identify classroom assessment techniques to assess CLOs

13. Upcoming events … CETL Reception TODAY from 3 to 5 in COB2 390
Next Faculty Academy meeting on May 9th at noon in COB2 290.
› Shall we discuss course evaluations? Ways to receive more responses and more useful feedback to act on them?

14. What is assessment (of student learning)?
1. Establish goals for student learning (expressed as learning outcomes)
2. Determine the evidence: the work students will do to demonstrate their learning
3. Design & provide intentional learning experiences (curriculum & pedagogy)
4. Gather & review evidence of student learning
5. Draw conclusions about student learning achievements in the aggregate
6. As necessary, act on the results to improve student achievement of learning goals
Assessment is a cyclical iterative process…
The gathering and reviewing of evidence of student learning is termed assessment, with for example, the term evaluation reserved for drawing a conclusion about the degree of student achievement.
Review cycle.
Then align Outcome: ability to write a technical report, a professional writing format
Evidence: the final two technical reports and how well students do… by end of semester, as well as improvement
Design: writing reports weekly guided by a rubric, provided detailed feedback and modeling consistent with rubric. Tracking students use of feedback.
Gather evidence and draw conclusions about student learning and act on results: mid-semester examines student improvements and notes that students who are using feedback are improving more than others. Doubles back and shared with this class to motivate use of feedback to improve.
LOs – what students know and are able to do as a result of a learning experience
Evidence – might be a lab report, exam question, paper, portfolio, a class presentation
Is the pedagogy part – what learning experiences will you as an instructor have the students engage in to develop the skills identified in the learning outcome and to be demonstrated in the evidence. How will you prepare them to succeed on that assignment/assessment/evidence? Peer review, strip sequence, - opportunities to practice with feedback. This might be informed by the literature, including disciplinary literature, on teaching strategies and pedagogy.
Gather evidence of student learning, assign and collect the assignment and review it
Draw conclusions about student learning as a group - did most students achieve the learning objective to the intended level of proficiency? If so, celebrate! If not, identify what areas need to be strengthened and identify some revisions to improve student outcomes. This might be…. Tweak your pedagogy, revise how you provide individual students with feedback to build skills, maybe it needs to be more targeted and specific (arguably part of pedagogy), might conclude that the evidence didn’t really get students to do what you wanted them to do, ex. the essay prompt didn’t get students to do what you wanted to see so revise (very common), or you might realize that the goal was too ambitious or not ambitious enough… revise it.
Activity… take the pieces
Hybrid of Suskie (2009) , the CIRTL Teaching-as-Research (TAR) framework, and Backward Design (McTighe &Williams, 1998)

15. Scholarship of Teaching & Learning
Describe Problem Related to Learning
Literature Search
Design
a Project:
3) Solution
Scholarly Teaching
Comparison, Context
Evidence:
4) Baseline
5) Assessment
SoTL Confirmed?
Knowledge Base of Teaching and Learning in Higher Ed.: The Literature
Propose
6) Presentation
7) Publication
Scholarship of
Teaching & Learning
Presentation
Publication
Peer Review
©Milton D. Cox