Chapter 4. Eye tracking as a measure of cognitive effort for post-editing of machine translation
-
Joss Moorkens
Abstract
The three measurements for post-editing effort as proposed by Krings (2001) have been adopted by many researchers in subsequent studies and publications. These measurements comprise temporal effort (the speed or productivity rate of post-editing, often measured in words per second at the segment level), technical effort (the number of actual edits performed by the post-editor, sometimes approximated using the Translation Edit Rate metric (Snover et al. 2006), again usually at the segment level), and cognitive effort. Cognitive effort has been measured using think-aloud protocols, pause measurement, and, increasingly, eye-tracking. This chapter provides a review of studies of post-editing effort using eye-tracking, noting the influence of publications by Danks et al. (1997), and O’Brien (2006, 2008), before describing a single study in detail. The detailed study examines whether predicted affort indicators affect post-editing effort and results were previously published as Moorkens et al. (2015). This chapter focuses instead on methodology and the logistics of running an eye-tracking study recording over 70 sessions. Most of the eye-tracking data analysed were unused in the previous publication, and the small amount presented was not described in detail due to space constraints. In this study average fixation count per segment correlates very strongly with temporal effort, and average fixation duration correlates strongly with technical effort, a result that we compare with other studies of post-editing effort.
Abstract
The three measurements for post-editing effort as proposed by Krings (2001) have been adopted by many researchers in subsequent studies and publications. These measurements comprise temporal effort (the speed or productivity rate of post-editing, often measured in words per second at the segment level), technical effort (the number of actual edits performed by the post-editor, sometimes approximated using the Translation Edit Rate metric (Snover et al. 2006), again usually at the segment level), and cognitive effort. Cognitive effort has been measured using think-aloud protocols, pause measurement, and, increasingly, eye-tracking. This chapter provides a review of studies of post-editing effort using eye-tracking, noting the influence of publications by Danks et al. (1997), and O’Brien (2006, 2008), before describing a single study in detail. The detailed study examines whether predicted affort indicators affect post-editing effort and results were previously published as Moorkens et al. (2015). This chapter focuses instead on methodology and the logistics of running an eye-tracking study recording over 70 sessions. Most of the eye-tracking data analysed were unused in the previous publication, and the small amount presented was not described in detail due to space constraints. In this study average fixation count per segment correlates very strongly with temporal effort, and average fixation duration correlates strongly with technical effort, a result that we compare with other studies of post-editing effort.
Chapters in this book
- Prelim pages i
- Table of contents v
- Chapter 1. Introduction 1
- Chapter 2. A mapping exercise 11
-
Part I. Method
- Chapter 3. Overcoming methodological challenges of eye tracking in the translation workplace 33
- Chapter 4. Eye tracking as a measure of cognitive effort for post-editing of machine translation 55
- Chapter 5. Analysing variable relationships and time-course data in eye-tracking studies of translation processes and products 71
-
Part II. Process
- Chapter 6. Recognition and characterization of translator attributes using sequences of fixations and keystrokes 97
- Chapter 7. Problem solving in the translation of linguistic metaphors from Chinese into Portuguese 121
- Chapter 8. Working styles of student translators in self-revision, other-revision and post-editing 145
- Chapter 9. Visual attention distribution in intralingual respeaking 185
-
Part III. Product
- Chapter 10. Subtitling of British stand-up comedy into Italian 205
- Chapter 11. Towards a quantitative measurement of equivalent effect and a tentative conceptualisation of cognitive equivalence 225
- Chapter 12. The impact of AVT mode on audience reception 259
- List of abbreviations 287
- Contributors 289
- Subject index 293
Chapters in this book
- Prelim pages i
- Table of contents v
- Chapter 1. Introduction 1
- Chapter 2. A mapping exercise 11
-
Part I. Method
- Chapter 3. Overcoming methodological challenges of eye tracking in the translation workplace 33
- Chapter 4. Eye tracking as a measure of cognitive effort for post-editing of machine translation 55
- Chapter 5. Analysing variable relationships and time-course data in eye-tracking studies of translation processes and products 71
-
Part II. Process
- Chapter 6. Recognition and characterization of translator attributes using sequences of fixations and keystrokes 97
- Chapter 7. Problem solving in the translation of linguistic metaphors from Chinese into Portuguese 121
- Chapter 8. Working styles of student translators in self-revision, other-revision and post-editing 145
- Chapter 9. Visual attention distribution in intralingual respeaking 185
-
Part III. Product
- Chapter 10. Subtitling of British stand-up comedy into Italian 205
- Chapter 11. Towards a quantitative measurement of equivalent effect and a tentative conceptualisation of cognitive equivalence 225
- Chapter 12. The impact of AVT mode on audience reception 259
- List of abbreviations 287
- Contributors 289
- Subject index 293
