1. Lynn Thompson Center for Applied Linguistics Startalk Network for Program Excellence Chicago, Illinois October 16-18, 2009 Formative and Summative Assessment

2. Overview I.CONTEXT – STARTALK Assessment Workshop II.What is Assessment? III.Key concepts IV.Formative and Summative Assessment V.Suggestions for evaluating assessment data 1

3. STARTALK Proficiency Assessment Training Program  CAL Proficiency Assessment Training Program Blended Course Online introduction to assessment principles 2-3 day workshop allowing for networking and test item development Online follow-up modules addressing specific needs for instructors and administrators Further online training options: Assessment Training Online (ATOL) Grades 9-16 Student Oral Proficiency Assessment (SOPA) Grades K-8

4. STAR Workshop  Purpose: Help participants learn about and plan for assessment in STARTALK summer programs  Audience: Instructors and Program Directors who work with STARTALK programs in Arabic, Chinese, Hindi, Persian, Turkish, and Urdu. 3 LanguageSummer 2008Summer 2009 Arabic38 Chinese169 Hindi/Urdu17 Persian20 Turkish01 Multiple04 Total:2229

5. 2008 STAR Workshop Content  Online Modules 1-4  (May 2-May 30)  Live Workshop  (June 8-19)  Online Modules 5-8  (June 13- December 2008) 4 ModuleTopics covered Module 1Introduction to workshop and online technology Module 2 Key assessment concepts Needs assessment Module 3 ACTFL Proficiency Guidelines Assessing the four skills Module 4Formative assessment Summative assessment ModuleTopics covered Live workshopNeeds assessment National Standards LinguaFolio Langnet Developing assessment tasks Planning for assessment ModuleTopics covered Module 5 Review live workshop Share materials Sample assessment tasks Module 6 Interpretive listening Interpretive reading Module 7 Computer-based assessment Student self-assessment Assessing culture Planning for assessment Module 8Rating and rubrics Interpreting results Teacher certification

6. Star Workshop 2009  Online modules  Choice between 2 live workshops (one in July and one in August)  Follow up modules  Training courses

7. Challenges  Range of assessment knowledge  LCTLs are less frequently taught to lower age groups – Lack of existing material that is developmentally appropriate – Instructors may not be accustomed to working with young learners, even if they have teaching experience

8. What is Assessment?  As instructors, we need to be able to track our students' language growth. Assessment is how we do this.  Assessment = to sit beside  Assessment = the opportunity to enhance, empower, and celebrate students’ learning while giving guidance to instructors.  Assessment = a collaboration between the student and the instructor: – monitor students' daily progress using formative assessment, – check in on student progress at key points in the school year using summative assessment.

9. Key Testing Concepts  Validity refers to the suitability of a test for your purposes. – What do you want to measure? – What will the results be used for? – Who is being tested? 8

10. Key Testing Concepts  Reliability is the consistency of test results. – Consistent results when tested on two separate occasions – Consistent scoring by different instructors  Reliability is closely tied to validity. A test can be reliable without being valid. However, a test cannot be valid unless it is reliable. 9

11. Impact  Who will be affected by the assessment results: parents, students, teachers, administrators, funders, and more. These people are all stakeholders.  Tests can have positive or negative effects (washback) 10

12. Stakeholders 11

13. Assessment and LCTLs  Assessing Less Commonly Taught Languages can be more difficult due to a smaller pool of assessment resources from which to draw.  When ready-made assessments are not available, instructors must develop their own assessment materials, making it even more important that they understand the principles of assessment.

14. Challenges in Language Assessment  Instructors must understand language assessment in order to participate in the language learning culture (Shepard, 2000).  High-stakes tests, like those that the government may use in hiring and assigning language specialists, must be valid and reliable. However, in many cases, such tests have not been developed for critical languages. 13

15. Discussion (1)  What challenges have you faced in finding and using assessments in your language?  How have you addressed them? 14

16. Types of Assessment  Formative assessment is used during the course of instruction to provide feedback to the teacher, learner, or both about the learner's progress toward desired educational outcomes.  The results of formative assessments are often used in planning subsequent instruction.

17. Examples of Formative Assessment  Formative assessments may take many forms, such as: – Quizzes and tests, – class presentations, – projects, – portfolios and – self assessments. 16

18. Formative Assessment  When you plan formative assessments, think about how you will relate the assessment to your instruction.  Assessment should reflect classroom activities and provide useful information to you and your students.  It is also important to consider how you will communicate results to your students. You may want to plan for time to explain results.

19. Summative Assessment  Summative assessment refers to assessments that are generally conducted at the end of a unit or at the end of a sequence of units to provide feedback to the teacher, learner, or both about the learner's achievement of course goals or proficiency.

20. Summative Assessment  The information gathered from summative assessments is often used to make decisions about:  grading,  program evaluation,  tracking or  accountability. 19

21. Summative Assessment  Summative assessment should reflect classroom instruction.  Summative assessment may have higher stakes attached to the results.  The stakes will determine what type of assessment is appropriate for the decisions you need to make. 20

22. Student Self Assessment  Student self-assessment is useful formative assessment, but may not be sufficient to assign grades for the semester or to determine if students have mastered an instructional unit.  A comprehensive test of all the material covered in a semester would provide a better basis for making grading decisions. 21

23. Evaluating Assessment Results  Classroom assessment activities allow you to collect information about what students are able to do.  In order to track student progress and inform instruction, it is necessary to have a way to evaluate the information that you have collected.

24. Examples of how to evaluate student progress  1) Checklists – Checklists can be used to record task completion in the classroom or if a student has learned particular language forms and functions. 23

25. Rating Scales  A rating scale provides a range of measures that can be selected for a student's performance.  One of the most common rating scales that we see in schools is the A-F grading scale.  Some schools also use rating scales to measure consistency or frequency 24 NeverSeldomSometimesAlways

26. Rubrics  Like rating scales, rubrics allow you to rate student performance and help you track their progress over time.  Rubrics define different levels of student performance and can also allow you to evaluate different aspects of a student's performance. 25

27. Rubric Power Move  Sharing the rubric you are developing with your students allows them to see what aspects they will be evaluated on and what each of the levels of performance require. 26

28. Discussion  What are your language assessment needs?  How do you go about finding language assessments when you need them? 27

29. Assessment Resources – Understanding Assessment: A Guide for Foreign Language Educators is a tutorial on language testing for foreign language educators. It is also an excellent resource for anyone who want to learn more about language testing http://www.cal.org/flad/tutorial/ http://www.cal.org/flad/tutorial/ – Foreign Language Assessment Directory is a searchable annotated database of K-16 foreign language assessments http://www.cal.org/CALWebDB/FLAD/ 28

30. 29 Shepard, L. (2000). The role of assessment in learning culture. Educational Researcher, 29.7, 4-17. Stiggins, G. (1997). Student Centered Classroom Assessment. Upper Saddle River, NJ: Prentice Hall. Stoynoff, S., & Chapelle, C. A. (2005). ESOL tests and testing: A resource for teachers and program administrators. Alexandria, VA: TESOL Publications. References