Transcript Holistic vs Analytical Assessment in Legal Translation
Holistic vs Analytical Assessment in Legal Translation
Carmen Valero-Garcés & Francisco Vigier – University of Alcalá Mary Phelan – Dublin City University
• Assessment in Translation Studies and Professional Practice • Introduction to HA – Research Study on HA in Legal Translation • Introduction to AA – Research Study on AA in Legal Translation • Conclusions
2
Assessment in Translation Studies and Professional Practice
• Underresearched area • Common problems in TQA (Williams 2009) – The evaluator – Level of target language rigour – Seriousness of errors – Sampling vs full-text assessment – Quantification of quality – TQA purpose
3
What is Holistic Assessment?
• The evaluator gives a TT a rating (0-10) or evaluative letter (e.g. A = excellent, B = very good) based on an overall impression • Frequently used in both academia and industry •
Advantages
less time-consuming and assessment of translations at the discourse/text level not at the sentence/word level (Garant 2009) • Some attempts of systematization (Waddington, 2001) •
Disadvantages
subjective, hence arbitrary, intuitive, unscientific, unsystematic and unreliable; does not provide a clear justification of the result (Waddington 2001) 4
Research Study on HA in Legal Translation
• Analyse strengths and weaknesses of holistic methods for the assessment of legal translation ( interrater reliability) • One of the WS1 essential documents translated into SP by a student on MA in Translation • That translation assessed numerically (0-10) by ten evaluators • Evaluators surveyed on their assessment method 5
6
• Numeric assessment
Results
7
Results (2)
• Survey – Most evaluators ranked pragmatic errors as those with highest relevance and linguistic errors as those with lowest relevance – Very different opinions expressed by respondents as to the translation’s strengths and weaknesses (i.e. “The message is
appropriately conveyed. It fulfills its communicative function”
vs. “Errors regarding sense, coherence, punctuation... A poor
quality translation”
assessment is based on personal criteria, thus subjective and variable 8
Analytical Assessment
ANALYTICAL ASSESSMENT
9
ATA • ATA system – (a) grid, (b) flowchart and (c) Explanation of Error Categories 10
ATA Grid
ATA CERTIFICATION PROGRAM FRAMEWORK FOR STANDARDIZED ERROR MARKING Exam Number: Exam Passage:
Version 2009
Check here if for Review 1 2 4 8 Errors that concern the form of the exam 16 Treat missing material within the passage as an omission Code UNF Translation/strategic/transfer errors: Negative impact on understanding/use of target text MT ILL IND MU A O T R F L FA COH AMB ST OTH Reason Unfinished (if a passage is substantially unfinished, do not grade the exam) Illegibility Indecision, gave more than one option Mistranslation (use a subcategory if possible) Misunderstanding of source text (if identifiable) Addition Omission Terminology, word choice Register Faithfulness Literalness Faux ami (false friend) Cohesion Ambiguity Style (inappropriate for specified type of text) Other (describe) Mechanical errors: Negative impact on overall quality of target text. Points may vary by language. Maximum 4 points G SYN Grammar Syntax (phrase/clause/sentence structure) P SP/CH D C WF/PS U OTH Punctuation Spelling/Character (usually 1 point, maximum 2, if more than 2 points, another category must apply) Diacritical marks/ Accents Capitalization Word form/ Part of speech Usage Other (describe) 0 0 x 2 =0 0 x 4 = 0 0 x 8 = 0 A grader may stop marking errors when the score reaches 46 error points Total error points (add column totals): 0 0 A grader may award a quality point for each of up to three specific instances of exceptional translation Quality points (maximum 3) 0 Column totals Quality points are subtracted from the error point total to yield a final score. A passage with a score of 18 or more points receives a grade of Fail. Final passage score (subtract quality points from error points) 0 11
ATA flowchart
12
• • UAH text – holistic - 532 words in ST DCU text – analytical – 256 words in ST • 5 assessors – three in Europe plus two ATA assessors 13
Assessor
Spanish evaluator 2 Spanish evaluator 3 Spanish evaluator 1 ATA evaluator 1 ATA evaluator 2
Evaluators’ Verdicts
Score
9 16 23 45+ 43
Verdict
Pass Pass Would accept it with reservations Fail Fail 14
Conclusions
• • • • HA: subjective method with a low degree of inter-rater reliability Cost and time efficiency HA as supplementary method for LT assessment?
AA: even though the system appears self explanatory, there is a lot of variation in the overall result.
AA: The ATA evaluators have years of experience of using this method.
15
References
Garant, M. (2009). A case for holistic assessment. AFinLA- e Soveltavan kielitieteen tutkimuksia 2009, 1, 5-17.
Waddington, C. (2001b). Should translations be assessed holistically or through error analysis? Hermes, Journal of Linguistics, 26, 15-38.
Williams, M. (2009). Translation Quality Assurance. Mutatis Mutandis, Vol 2, No 1., 3-23 16
Carmen Valero-Garcés Francisco Vigier Mary Phelan