INDICATORS FOR PRAGMATIC INSTRUCTION Some quantitative tools

THIS STUDY DISCUSSES THREE METHODS OF ASSESSING PRAGMATIC PRODUCTION BY JAPANESE LEARNERS OF ENGLISH AS SL 1. Written discourse completion test 2. Language lab discourse completion test tape records 3. Role-play scenarios

ANALYZED SPEECH ACTS  Request  Refusals  Apologies These are the most researched speech act in the literature and appropriate for contrasting speech-act realization across a number of languages.

VARIABLES  Power (P)  Social distance (D)  Degree of imposition (R) These are identified as the three independent and culturally sensitive variables that subsume all other variables and play an important role in speech-act behavior.

Relative power : the degree to which the speaker can impose his or her will on the hearer. This then relates to the relative rank, title, or social position between the two interactants.

Social distance: the distance between the speaker and the hearer, the degree of familiarity and solidarity they share as represented through in-group or out-group membership.

Absolute ranking of imposition: this is the potential imposition of carrying out the speech act, in terms of the expenditure of goods and/or services by the hearer, or the obligation of the speaker to perform the act.

The categories adopted for evaluating the speaker’s actual responses Correctness of linguistic expression Language correctness Amount of information Formality Social aspect of language Directness Politeness They provide the basis for a scale measuring the pragmatic competence.

RATIONALE FOR DEVELOPING DIFFERENT INSTRUMENTS To determine the potential differential effectiveness and utility of instruments. Some methods may assess the interested linguistic component better than others.

RESEARCH QUESTIONS 1. To what extent do the different test methods affect performance ratings? 2. To what extent would the different speech acts differ in terms of the mean score? 3. What are the correlations between the DCT, language lab, and role-play? 4. To what extent would the different aspects of speech act, expressions, amount, formality, directness, or politeness be rated differently? 5. What is the effect for power, distance, and degree of imposition on scores across the different test types?

DEVELOPMENT OF INSTRUMENTS  No gender specification in the task description  Face to face situations  ‘you want’ or ‘you need’ used in the prompts for requesting  No context for apologies resulted from physical contact, injury, or violating social norms  No socially stigmatized roles (famous star,patron)  Professionally defined or formulaic patterns are avoided ( doctor-patient, lawyer- client)

THREE STAGES OF DEVELOPING THE INSTUMENTS  Writing  Piloting  Revising

PARTICIPANTS  25 Japanese students in intensive English language program.  8 males and 18 females between the ages from 21 to 42.  Pre-university level, TOEFL below 500.  Groups homogeneity of language ability will prove problematic later.

MATERIALS AND PROCEDURES MaterialsProcedureEvaluation Lab DCTTape-recorded By 3 trained raters/ likert type Open-ended DCT At home, in a short time Role-playVideo-recorded

RESULTS  There was a little variance due to the homogeneity of the group.  Thus, the data was negatively skewed.  ANOVA indicated a significant difference between DCT and Lab DCT, and Lab DCT and role-play.  Only in refusals and apologies a significant effect was found. Apologies are rated slightly higher than the others due to its formulaic nature.

To what extent do the different test methods affect performance ratings? Lab DCT appeared to be slightly more difficult than the others. To what extent would the different speech acts differ in terms of the mean score? Only in refusals and apologies a significant effect was found. Apologies are rated slightly higher than the others due to its formulaic nature.

What are the correlations between the DCT, language lab, and role-play? MethodsDCTLANG. LABROLE-PLAY DCT1.000 LANG. LAB ROLE-PLAY Correlation for the different methods

LAB DCT Direction: you will hear descriptions of 24 different situations. Each description will be repeated once. After the description, write what would you say if you were in the situation described. Situation 1. You are applying for a job. You go to the office to run in your application form to the manager. You talk to the manager for a few minutes. When you move to give your form to the manager, you accidentally knock over a vase on the desk and spill water over a pile of papers. You:_____________________________

DCT Directions: Read each of the following scenarios. In the space provided, write what you would say in the situation in a normal conversation. Situation 1. You live in a large house. You hold the lease to the house and rent out the other rooms. You are in the room of one of your housemates collecting the rent. You reach to take the rent check when you accidentally knock over a small, empty vase on the desk. It doesn’t break. You:____________________________________

ROLE-PLAYS Directions: there are 8 role-plays. You will be given a role card before each role play begins. The role card will describe the situation and your role. If you have questions about the situations and your role, please ask before role play begins. During some of the role-plays, you will be given another role card with further instructions.

Situation 1. Background One: Last week you had trouble with your company van and took it to the company mechanic. The mechanic promised to have it ready by tomorrow at noon. However, you just found out that you have to go on a business trip tomorrow morning and have lots of display materials and samples to bring with you. So you need your van to be ready early tomorrow morning. Now: you go over to the shop and walk over to the head mechanic who is eating lunch.

Background two: when the mechanic stepped away, you accidentally knocked over his coffee, which spilled over some paperwork on the table. Now: the mechanic comes back.

To what extent would the different aspects of speech act, expressions, amount, formality, directness, or politeness be rated differently? Analysis showed a significant effect but not a significant difference between the pragmatic components. The 2 components most closely linked to the grammatical and lexical areas (amount and expression) were the most difficult ones.

What is the effect for power, distance, and degree of imposition on scores across the different test types? Due to the homogeneity, the differences are slight. Degree of imposition create a burden for the speaker in negotiating an acceptable answer.

DCT and Role-play formats should be used, as Lab DCT was the least rated. Apologies are less demanding than refusals. The more grammatical and lexical variables were rated more harshly than the social ones. High degree of imposition created a difficulty for the speaker in the role-plays.