1. Holistic vs Analytical Assessment in Legal Translation Carmen Valero-Garcés & Francisco Vigier – University of Alcalá Mary Phelan – Dublin City University

2. Assessment in Translation Studies and Professional Practice Introduction to HA – Research Study on HA in Legal Translation Introduction to AA – Research Study on AA in Legal Translation Conclusions 2

3. Assessment in Translation Studies and Professional Practice Underresearched area Common problems in TQA (Williams 2009) – The evaluator – Level of target language rigour – Seriousness of errors – Sampling vs full-text assessment – Quantification of quality – TQA purpose 3

4. What is Holistic Assessment? The evaluator gives a TT a rating (0-10) or evaluative letter (e.g. A = excellent, B = very good) based on an overall impression Frequently used in both academia and industry Advantages  less time-consuming and assessment of translations at the discourse/text level not at the sentence/word level (Garant 2009) Some attempts of systematization (Waddington, 2001) Disadvantages  subjective, hence arbitrary, intuitive, unscientific, unsystematic and unreliable; does not provide a clear justification of the result (Waddington 2001) 4

5. Research Study on HA in Legal Translation Analyse strengths and weaknesses of holistic methods for the assessment of legal translation (  interrater reliability) One of the WS1 essential documents translated into SP by a student on MA in Translation That translation assessed numerically (0-10) by ten evaluators Evaluators surveyed on their assessment method 5

6. 6

7. Results Numeric assessment 7

8. Results (2) Survey – Most evaluators ranked pragmatic errors as those with highest relevance and linguistic errors as those with lowest relevance – Very different opinions expressed by respondents as to the translation’s strengths and weaknesses (i.e. “The message is appropriately conveyed. It fulfills its communicative function” vs. “Errors regarding sense, coherence, punctuation... A poor quality translation”  assessment is based on personal criteria, thus subjective and variable 8

9. ANALYTICAL ASSESSMENT Analytical Assessment 9

10. ATA ATA system – (a) grid, (b) flowchart and (c) Explanation of Error Categories 10

11. ATA Grid 11

12. ATA flowchart 12

13. UAH text – holistic - 532 words in ST DCU text – analytical – 256 words in ST 5 assessors – three in Europe plus two ATA assessors 13

14. Evaluators’ Verdicts AssessorScoreVerdict Spanish evaluator 29Pass Spanish evaluator 316Pass Spanish evaluator 123Would accept it with reservations ATA evaluator 145+Fail ATA evaluator 243Fail 14

15. Conclusions HA: subjective method with a low degree of inter-rater reliability Cost and time efficiency  HA as supplementary method for LT assessment? AA: even though the system appears self- explanatory, there is a lot of variation in the overall result. AA: The ATA evaluators have years of experience of using this method. 15

16. References Garant, M. (2009). A case for holistic assessment. AFinLA- e Soveltavan kielitieteen tutkimuksia 2009, 1, 5-17. Waddington, C. (2001b). Should translations be assessed holistically or through error analysis? Hermes, Journal of Linguistics, 26, 15-38. Williams, M. (2009). Translation Quality Assurance. Mutatis Mutandis, Vol 2, No 1., 3-23 16

17. Thank you! Carmen Valero-Garcés carmen.valero@uah.es Francisco Vigierfrancisco.vigier@uah.es Mary Phelanmary.phelan@dcu.ie