The Effectiveness of Corrective Feedback on Chinese EFL Writers’ Grammatical Accuracy Improvement Dongmei Cheng Northern Arizona University

Motivation No general conclusion on the efficacy of grammatical corrective feedback (e.g. Truscott, 1996, 2007; Ferris, 1997, 1999, 2004, 2006)

Motivation No consensus on how explicit feedback should be: – More explicit feedback: Providing metalinguistic information helps (Bitchener, 2008; Bitchener et al, 2005; Lalande, 1982) – Less explicit feedback: Learners only need guidance in locating the errors (Chandler, 2003; Lee, 1997; Robb, Ross & Shortreed, 1986)

Motivation Few studies has truly adopted “a sustained, systematic, replicable manner that would allow for comparisons across similar or different contexts and student populations” (Ferris, 2004, p.55).

Motivation There’s a need to adopt a controlled design, which has a comparable selection of – corrective feedback types – targeted error types – Writing tasks

Research Questions RQ: Does two types of indirect corrective feedback help to reduce the amount of targeted grammatical errors in students’ argumentative writings over time? If both types of feedback help, which one is more effective? Feedback type 1: Highlighting the targeted errors Feedback type 2: Highlighting the targeted errors and providing abbreviated error codes

Research Questions Time (Within two months) – Time 1: First Draft of Essay 1 (Week Three) – Time 2: Revised Draft of Essay 1 (Week Five) – Time 3: First Draft of Essay 2 (Week Eight)

Participants Total participants N=95 (Female N=60; Male N=35) Participants came from three College English classes taught by the same EFL instructor in a big public university in Northern China. (original participant pool N=120)

Participants All college freshmen from three major areas: Administrative Management, Economics and International Accounting. Have no regular English writing practices and multiple-draft writing experiences. Are generally competent English readers but relatively weak in speaking and writing.

Participants AgeCollege Entrance Exam Score Years of English Learning Mean Standard Deviation

Design Two experimental groups + A control group All learners in the three classes (initial participants’ pool N=120) were randomly assigned to three different types of feedback: – Highlighting the targeted errors (Highlight) – Highlighting the targeted errors with abbreviated codes (Code) – No grammatical feedback (None)

Procedure All participants submitted their first essays electronically through the school online course platform. The EFL instructor downloaded the essays and sent them to the researcher. The researcher provided grammatical feedback using the “highlighter” and “comment” function on Microsoft Word. The researcher sent back the grammatically commented essays to the EFL instructor. The EFL instructor added in brief content and organization feedback for all participants (including the control group)

Variables Independent variables: – Feedback type (Highlight, Code vs. None) – Time (Essay 1; Revised Essay 1 vs. Essay 2) Dependent variable: Targeted error score (per essay)= occurrence of targeted error(s)*100 total word

Argumentative Writing Tasks Two essay topics from the IELTS Task Two Writing prompt pool. Selection criteria: – Participants are familiar with both topics. – The two prompts fit in the “agree vs. disagree” type, which provides a claim and asks the participants to what extent they agree or disagree with the given claim.

Argumentative Writing Tasks Essay 1: More and more women go out to work. Is it the government’s responsibility to subsidize them and provide free staff and facilities to care for their children? To what extent do you agree or disagree to this idea?

Argumentative Writing Tasks Essay 2: The government is responsible for protecting a nation’s cultural identity. Thus, some people believe new buildings should be built in traditional styles. To what extent do you agree or disagree with this opinion?

Targeted Errors—Type I Type I: Noun endings (NE) e.g. This phenomenons become more and more common among office ladies. More and more woman go out to work.

Targeted Errors—Type II Type II: Verbs – Subject-Verb Agreement (SV) The government have responsibility to subsidize them. Culture exist in every one’s spirit. – Verb Tense (VT) As the society developed, more people want their lives to be more comfortable and convenient. To protect or rebuilt old buildings of traditional styles is more meaningful than build some ones.

Targeted Errors—Type II Type II: Verbs – Subject-Verb Agreement (SV) – Verb Tense (VT) – Verb Form (VF) If new buildings be built in traditional styles, people will pay more cost and can’t adapt to live in these buildings. In order to protect the nation’s culture, what we really need to do is to protect the treasure we have already own, like old buildings, old paintings and so on.

Targeted Errors—Type III Type III: Sentence Structure – Sentence Fragment (SF) Here^ my reasons. So the government should subsidize the women. Because now people have a lot of pressure of modern life. And finding job is easier.

Targeted Errors—Type III Type III: Sentence Structure – Sentence Fragment (SF) – Run-on Sentences (RO): (including comma errors) There is a woman go to work. Some problems appeared, who will care for their children. A series of traditions are left which have a profound effect on us.

Targeted Errors—Type IV Type IV: Word Form – Wrong word formations including nouns, adjectives and adverbs (WF) The construct which represent a nation’s history are of vital importance. Mothers always show their children with patient. When they get home, they also need to work hard to keep the room cleanly.

Reliability Check Intra-rater reliability: 25% of the essays were coded again by the researcher after two months. Pearson Correlation r=.97 Inter-rater reliability: 25% of the essays were coded by a second coder. Pearson Correlation r=.91

Results—Descriptive Statistics Raw Error Score Essay 1Revised Essay 1 Essay 2 HighlightM=5.74 SD=2.51 M=2.74 SD=2.39 M=5.29 SD=3.33 CodeM=5.74 SD=2.90 M=1.19 SD=1.38 M=3.48 SD=2.32 NoneM=7.23 SD=3.43 M=6.30 SD=3.23 M=7.30 SD=3.98

Raw Error Occurrences

Results—Descriptive Statistics Normed Error Score Essay 1Revised Essay 1 Essay 2 HighlightM=3.69 SD=1.88 M=1.57 SD=1.40 M=3.29 SD=2.79 CodeM=3.54 SD=1.62 M=0.70 SD=0.86 M=2.07 SD=1.46 NoneM=3.03 SD=1.52 M=2.72 SD=1.29 M=3.05 SD=1.48

Normed Error Scores

Results—Group One (Highlight) Alpha was set as.05. Test: Within-subject repeated measure ANOVA RQ1: Does one type of indirect feedback (Highlight) help to reduce the error scores in students’ essays over time?

Results—Group One (Highlight) F(2)=14.97, p<.05 There is a significant time effect on the highlighting group’s error scores. Results from pairwise comparisons: – Essay1 vs RevEssay1: p<.05 – Essay1 vs Essay2: p>.05

Results—Group Two (Code) RQ2: Does another type of indirect feedback (code) help to reduce the error scores in students’ essays over time? Results from repeated measure ANOVA: F(2)=33.58 p<.05

Results—Group Two (Code) There is a significant time effect on the error scores of participants’ who received coded feedback. Pairwise comparison results: – Essay1 vs. RevEssay1: p<.05 – Essay1 vs. Essay2: p<.05

Results—Control Group Repeated ANOVA results: F(2)=.788, p>.05 There is no significant time effect on control group’s error scores.

Results—Between-subject Test: Between-subject repeated measure ANOVA Results: F(2)=4.40, p<.05 There is a significant difference in the error scores over times among the three groups.

Results—Between-subject Pairwise comparisons: – Highlight vs. Code: p<.05 – Highlight vs. None: p>.05 – Code vs. None: p<.05 There’s a significant difference on error scores over time between the two experimental groups. The “Highlight” group showed NO significant difference on error scores over time compared to the control group. The “Code” group showed a significant difference on error scores over time compared to the control group.

Error Scores from the Three groups Over time

Conclusion Both types of indirect feedback helped to reduce the targeted errors in students’ argumentative essays significantly in the short term (Essay1  RevEssay1) The more explicit feedback (Code) helped to reduce the targeted errors significantly from Essay 1 to Essay 2 (in a longer term) Highlighting the targeted errors and providing abbreviated codes had a more significant effect (compared to only highlighting the errors) on error reduction in students’ essays over time.

Pedagogical Implications EFL students could potentially benefit from more explicit corrective feedback on their argumentative writings. Providing grammatical codes helps the students to notice and understand the nature of their grammatical errors in the essay. A mutual understanding on the meanings of the error codes needs to be established beforehand. (e.g. an error chart)

Mean Error Scores for Noun Ending Errors

Mean Error Scores for Verb Errors

Mean Error Scores for Sentence Structure Errors

Mean Error Scores for Word Form Errors

Results for Word Form Errors Within-subject repeated measure ANOVA – Highlight: F(2)=.039, p>.05  no sig time effect – Code: F(2)=4.34, p<.05  sig time effect – Control: F(2)=18.54, p<.05  sig time effect Between-subject repeated measure ANOVA – F(2)=2.51, p>.05  no sig different difference among the effect of the three feedback types on the word form error scores

Implications of the unexpected findings on word form errors Learners perhaps have difficulties correctly interpret the feedback on word form errors. Compared to the other three types of targeted errors, word form errors might be relatively easier to be ignored by the learners, despite the provision of feedback. More explicit classroom instruction needs to be done to address the correct use of word forms in context (vocabulary instruction).