2. Assessing Grammar ChristineAliceStaceyJennyAnn

3. Test Evaluation Summary Teaching Summary Teaching Evaluation of an Existing Test Evaluation of an Existing Test

4. Five characteristics to measure communicative grammar 1. The test must provide more context than only a single sentence. 2. The test taker should understand what the communicative purpose of the task is. 3. He or she should also know who the intended audience is. 4. He or she must have to focus on meaning and not form to answer correctly. 5. Recognize is not sufficient. The test taker must be able “ to produce grammatical responses. ” By: P.R. Dickins

5. Therefore… Have the test taker say or write something (5) of discourse length (1) in order to perform some communicative task (2) for a known audience (3), and what is said or written must make sense (4). Have the test taker say or write something (5) of discourse length (1) in order to perform some communicative task (2) for a known audience (3), and what is said or written must make sense (4). Assessing grammar can only be tested as part of a test of writing or speaking? Assessing grammar can only be tested as part of a test of writing or speaking?

6. Important Principle: What are we trying to Measure? “ English grammar is chiefly a system of syntax that decides the order and patterns in which words are arranged into sentences. ” – Close 1982 Separated but closely related to assessing writing and assessing speaking. Separated but closely related to assessing writing and assessing speaking. Oral performanceOral performance Writing samplesWriting samples Accuracy and fluency Accuracy and fluency Count the errors and rate their severity Count the errors and rate their severity Focus on Grammar only Focus on Grammar only ‘ Meaning ’ is a separated matter‘ Meaning ’ is a separated matter Integrated testsIntegrated tests Discrete point testsDiscrete point tests

7. Traditional Paper and Pencil Test NRT: TOEFL NRT: TOEFL CRT: Classroom tests CRT: Classroom tests Types of Questions: Types of Questions: Complete sentencesComplete sentences Sentence combiningSentence combining Paragraph editingParagraph editing Correct topic sentence taskCorrect topic sentence task Multiple ChoiceMultiple Choice Fill in the blankFill in the blank True/FalseTrue/False CombinationCombination

8. Alternative Assessment Games Games Class observation Class observation Online activities Online activities Portfolio Portfolio

9. Games Design to restrict the language need Design to restrict the language need Usually these games are defined by their focus on Usually these games are defined by their focus on The Use of LanguageThe Use of Language Limited options for communicationLimited options for communication AccuracyAccuracy

10. Split Sentences Write out some sentences, and then cut each sentence in half. Hand out these pieces to the students, and ask them to find the matching half among the other students. Write out some sentences, and then cut each sentence in half. Hand out these pieces to the students, and ask them to find the matching half among the other students. Students ’ abilities in combing sentences Students ’ abilities in combing sentences Grammar knowledge to all forms of the sentences Grammar knowledge to all forms of the sentences

11. Examples If you eat that If you eat that If you touch the dog If you touch the dog If you steal my boyfriend If you steal my boyfriend If you go out now If you go out now If you don ’ t leave If you don ’ t leave If you don ’ t book a ticket If you don ’ t book a ticket you ’ ll be sick you ’ ll be sick it ’ ll bite you it ’ ll bite you I ’ ll never speak to you again I ’ ll never speak to you again you ’ ll get soaked you ’ ll get soaked I ’ ll call the police I ’ ll call the police you ’ ll be lucky to get a seat you ’ ll be lucky to get a seat

12. Sentences from pictures Hand out a sheet of pictures. Ask students to come out sentences from the pictures. Or ask students to tell a stories. Hand out a sheet of pictures. Ask students to come out sentences from the pictures. Or ask students to tell a stories. Enough grammar knowledge to make sentences by students. Enough grammar knowledge to make sentences by students. Accuracy Accuracy

14. Memory Test Give limited time for students to see the picture. Ask them to describe the picture without seeing it. Give limited time for students to see the picture. Ask them to describe the picture without seeing it. Testing students ’ ability of making sentences.(enough grammar knowledge) Testing students ’ ability of making sentences.(enough grammar knowledge) Accuracy Accuracy The transformation of students ’ sentences The transformation of students ’ sentences

16. Picture Dictation A student try to describe the picture to other students who haven ’ t seen it. A student try to describe the picture to other students who haven ’ t seen it. Testing if a student can make sentences which can help him/her communicate well. Testing if a student can make sentences which can help him/her communicate well. Accuracy Accuracy

18. Strip Story Make students into groups and assign them a text from a strip story. Ask them to find the order and resolve the problem from the story. Make students into groups and assign them a text from a strip story. Ask them to find the order and resolve the problem from the story. See if students can find the time order from the tenses of the sentences. See if students can find the time order from the tenses of the sentences. Accuracy Accuracy

20. Miming an action Ask students to role play in a limited situation. Ask students to role play in a limited situation. Through acting, it can test students ’ ability of thinking of lines. (which may contains different tenses.) Through acting, it can test students ’ ability of thinking of lines. (which may contains different tenses.) Accuracy Accuracy

21. Growing Stories Story-building activities Story-building activities Students ’ ability of making sentences with different tenses Students ’ ability of making sentences with different tenses

23. Questionnaires Turn what have taught in class into a questionnaire. Get students to survey each other. (does not contain fully-written-out questions.) Turn what have taught in class into a questionnaire. Get students to survey each other. (does not contain fully-written-out questions.) It can see if students ’ totally understand what have learned in class. Also it can see if students ’ can use it well. It can see if students ’ totally understand what have learned in class. Also it can see if students ’ can use it well.

25. Classroom Observation Classroom activities Classroom activities Classroom communication: Elicited dialogues Classroom communication: Elicited dialogues Written exercise Written exercise

26. Online Activities In multimedia teaching times, teachers can use Internet as a way of assessing students ’ ability of grammar In multimedia teaching times, teachers can use Internet as a way of assessing students ’ ability of grammar On-line quizzes help students to have more chances of practices On-line quizzes help students to have more chances of practices On-line tests also provide clear explanation after finishing the quizzes On-line tests also provide clear explanation after finishing the quizzes Students can surf on these grammar website to do practice Students can surf on these grammar website to do practice Teachers can ask students to do these exercise and to keep record for themselves. Finally report back for one of the reference of assessment Teachers can ask students to do these exercise and to keep record for themselves. Finally report back for one of the reference of assessment

27. Online activity : websites http://www.ccc.commnet.edu/grammar/index.ht m http://www.ccc.commnet.edu/grammar/index.ht m http://www.ccc.commnet.edu/grammar/index.ht m http://www.ccc.commnet.edu/grammar/index.ht m Ex1. Irregular verbs quiz http://www.ccc.commnet.edu/cgi- shl/par2_quiz.pl/irregular_quiz.htm Ex1. Irregular verbs quiz http://www.ccc.commnet.edu/cgi- shl/par2_quiz.pl/irregular_quiz.htm http://www.ccc.commnet.edu/cgi- shl/par2_quiz.pl/irregular_quiz.htm http://www.ccc.commnet.edu/cgi- shl/par2_quiz.pl/irregular_quiz.htm Grammar Assessment http://www.prenhall.com/grammarassessment/ Grammar Assessment http://www.prenhall.com/grammarassessment/ http://www.prenhall.com/grammarassessment/ Self-study grammar quizzes http://a4esl.org/q/h/grammar.html Self-study grammar quizzes http://a4esl.org/q/h/grammar.html http://a4esl.org/q/h/grammar.html ESL Quizzes Center http://www.pacificnet.net/%7Esperling/quiz/gra mmar ESL Quizzes Center http://www.pacificnet.net/%7Esperling/quiz/gra mmar http://www.pacificnet.net/%7Esperling/quiz/gra mmar http://www.pacificnet.net/%7Esperling/quiz/gra mmar

28. Portfolio Definition:a purposeful collection of student work that tells the viewer about the student Definition:a purposeful collection of student work that tells the viewer about the student Works should be organized by chronological order and category – teacher can evaluate the child ’ s achievements Works should be organized by chronological order and category – teacher can evaluate the child ’ s achievements Works including: Works including: 1. Traditional paper and pencil tests 1. Traditional paper and pencil tests 2. Classroom observation 2. Classroom observation 3. Online activities 3. Online activities 4. Written assignments 4. Written assignmentsWritten assignmentsWritten assignments

29. Written assignments Give students some assignments to practice at home Give students some assignments to practice at home Ask students using learned grammar in class to write journal Ask students using learned grammar in class to write journal Term paper throughout one semester including several drafts – we can see student ’ s progress in grammar. Term paper throughout one semester including several drafts – we can see student ’ s progress in grammar.

30. Advantages of portfolio Why portfolio is the best way to assess? Why portfolio is the best way to assess? Strengthening students ’ learning Strengthening students ’ learning Enhancing the teacher ’ s role Enhancing the teacher ’ s role Improving processes Improving processes

31. Test Evaluation of An Existing Test Section 2 of the Computer-Based Test of TOEFL Section 2 of the Computer-Based Test of TOEFL  General Background Information 1.Title: Section 2 of TOEFL-Structure and Written Expression 2.Author: Education Testing Service (ETS)

32. Test Evaluation of An Existing Test 3. Purpose of decision : Section 2 measures the ability to recognize language that is appropriate for standard written English. Section 2 measures the ability to recognize language that is appropriate for standard written English.

33. Test Evaluation of An Existing Test 4. Date of Application: a. 1962, the formation of a National Council on the TOEFL a. 1962, the formation of a National Council on the TOEFL b. July 1998 (Most parts of the world) b. July 1998 (Most parts of the world) c. October 2000 (Asia, including Taiwan) c. October 2000 (Asia, including Taiwan)

34. Practical Orientation 1.Ease of administration: a.Computers needed b.Convenience 2.Price: expensive ($130) 3.Quality of items: clarity

35. Practical Orientation 4.Ease of scoring: a.Photo score reporting b.Computerized scoring (0~30) c.Cumulative scoring 5.Ease of interpretation: transmission speed.

36. Practical Orientation 6.Test construction: a.Computer-adaptive Testing b.Complete an incomplete sentence c.Identify one unacceptable answer form four underlined words. 7.Timing: 20-25 questions/ 15-20 minutes (including “ pretest questions ” ) “ Description of the Computer-Based TOEFL Test ” www.ets.org

37. Test Evaluation of An Existing Test  Model Test

38. Analysis of TOEFL Grammar Test 8 sentence patterns 8 sentence patterns Basic Sentence & Compound Sentence Basic Sentence & Compound Sentence Adjective Clauses Adjective Clauses Participial Phrases Participial Phrases Appositive Appositive Noun Clauses Noun Clauses Adverbs Clauses Adverbs Clauses Prepositional Phrases Prepositional Phrases Expletives Expletives 方有毅 托福文法速成 來欣留學英語中心

39. Analysis of TOEFL Grammar Test 15 Answering Techniques 15 Answering Techniques 1. Word Form ………....20 % 2. Word Choice ……...14 % 2. Word Choice ……....14 % 3. Verbs ………………....11 % 4. Parallelism ………......9 % 5. Pronouns ……………...7 % 6. Singular/Plural Nouns …... ……... ……..6 % 7. Verbal (Infinitives, Gerunds, and Participles) …….. …...6 % 8. Prepositions …….. …..6% 9. Articles ……………………….5 % 10. Comparatives and Superlative sentence …… 4 % Superlative sentence …… 4 % 11. Conjunctions ……………… 4 % 12. Redundancy ………………….1 % 13. Negative Words …………… 2 % 14. Word Order ………………….4 % 15. Misplaced Modifiers …. …..1 % Total ……………………….100 % Total ……………………….100 % 方有毅 托福文法速成 來欣留學英語中心

40. Analysis of TOEFL Grammar Test Reliabilities: Reliabilities: describes the tendency of a set of scores to be ordered identically on two or more tests, and it can be estimated by a variety of statistical procedures. Table 1. Reliabilities and Standard Errors of Measurement (SEM) July 1998 – June 1999 ReliabilitySEM Listening0.982.76 Structure/Writing0.884.89 Reading0.882.73 Total Score0.9510.8 “ Description of the Computer-Based TOEFL Test ” www.ets.org

41. Analysis of TOEFL Grammar Test Validity Validity 1. ------pure lead, the lead ore is mined, then melted, and finally refined. (A) Obtaining (B) Being Obtained (A) Obtaining (B) Being Obtained (C) To obtain (D) It is obtained (C) To obtain (D) It is obtained 2. Louisa May Alcott is mainly noted for her very popular A B A B novel, Little Women, whom she published in 1868. novel, Little Women, whom she published in 1868. C D C D 3. Some hangars, buildings used to hold large aircraft, are A very tall that rain occasionally falls from clouds that form very tall that rain occasionally falls from clouds that form B C B C along the ceilings. ( 聶群 精修文法 來欣留學英語中心 ) along the ceilings. ( 聶群 精修文法 來欣留學英語中心 ) D

42. Analysis of TOEFL Grammar Test Strength: Accurate and objective scores Accurate and objective scores Careful monitoring of test Careful monitoring of test Commitment to international education Commitment to international education Expert advisory committees Expert advisory committees Highly skilled test developers Highly skilled test developers Longstanding reliability and consistency Longstanding reliability and consistency Ongoing test improvements Ongoing test improvements Standardized delivery procedures Standardized delivery procedures “ Description of the Computer-Based TOEFL Test ” www.ets.org

43. Analysis of TOEFL Grammar Test Conclusion Conclusion No other test in the world is as reliable as a standard for measuring a nonnative speaker ’ s ability to use English at the university level. No other test in the world is as reliable as a standard for measuring a nonnative speaker ’ s ability to use English at the university level. “ Description of the Computer-Based TOEFL Test ” www.ets.org

44. END