Spotlight on the reader: methodological challenges in combining translation process, product, and translation reception

2.1.1 The choice of proxies for cognitive effort

At the heart of methodological challenges in combining cognitive effort studies in translation and reception lies identifying valid and reliable measures capable of gauging cognitive effort invested in a translation task by the translator and in a reading task by the TT reader. The selection of appropriate measures is based on how cognitive effort is defined and operationalised.

Across RS, cognitive effort is often understood as attention or mental-resource allocation, cognitive load, processing difficulty, fluency disruption, and engagement in higher-order processing (e.g., Alter and Oppenheimer 2009; Kahnemann 1973; Sweller 2011). In CTIS, it remains a central construct, often conceptualised as the amount of mental resources exerted to produce a translation (e.g., Hvelplund 2011; Krings 2001; Vieira 2014). Using a broader understanding, the cognitive effort of the translator may be defined as “[t]he total effort that the translator expends during the translation task [and] the target text (TT) is then the product of this translator effort” (Hunziker Heeb 2020: 48), and researched analysing its different representations as indicators of the same cognitive effort, rather than technical, temporal, or cognitive efforts (see Hunziker Heeb 2020; Pietryga 2025).

Cognitive effort invested in performing a task by the translator and the reader lends itself to indirect measurement with ‘online’ verbal descriptions of thoughts (TAPs), retrospective rating scales (evaluating the subjective feelings of effort), comprehension accuracy (more inaccurate answers on text comprehension questions indexing high cognitive effort), brain waves (changes in the selected alpha or theta waves indexing changes in cognitive effort), physiological signals, reading times (longer time indexing higher effort), and eye movements – higher values of measures such as a number of fixations, regressions, and dwell time indexing higher effort (Gile and Lei 2020: 269–273). Its distribution across translation tasks is also investigated with hesitation pauses (Lacruz and Shreve 2014; O’Brien 2006) and eye-key span (Dragsted and Hansen 2008; Pietryga 2025), with longer pauses and longer time lags taken to reflect enhanced cognitive effort. Within CTIS, data registered with keystroke logging, screen-capture techniques, and eyetracking – today often used in triangulated and mixed-methods designs – are further interpreted within frameworks mainly developed for post-editing such as Krings’s (2001) temporal, technical, and cognitive indicators.

Eye-movement control models in reading (e.g., the E-Z Reader model) implicitly assume that “the link between eye movements and cognition is quite tight” (Reichle et al. 2003: 511), and that the distribution of visual attention aligns with information processing (the eye-mind hypothesis, Just and Carpenter 1980). To analyse this distribution across the tasks involving reading, researchers readily apply Rayner’s (2009) framework, where specific eye-movement measures are assumed to reflect specific stages of processing during reading. Early-stage eye-movement measures are taken to reflect initial lexical access (low-order visual encoding and initial lexical processing) and primarily include first-pass fixation duration, gaze duration (sum of all first-pass fixation durations), and saccade length. Late-stage eye-movement measures, assumed to reflect post-lexical access (higher-order processes thereof: semantic integration, comprehension), include regressions and re-reading time, and number of passes (runs).

Dwell time (total fixation duration) – a more global measure – reflects the total cognitive effort exerted over a specific AOI (Walker 2021). Although the exact labels for eyetracking measures vary, a prevailing view emerges: their higher values recorded to AOIs (a word, phrase, sentence, excerpt, or whole text) are taken as proxies for the increased cognitive effort invested in processing the AOIs. The research hypothesis will guide the choice of measure (early, late, or global) that is relevant to addressing the research question.

2.1.2 The choice of experimental stimuli to gauge cognitive effort

Another challenge of integrating translation process, product, and reception scopes lies in the choice of experimental stimuli. When the same experimental material (e.g., a text) is utilised in a translation-process-and-product study and the subsequent reading study, many decisions are mutually dependent and thus become more complex. Cognitive effort is sensitive to the lexico-semantic, syntactic, and stylistic features of the text (Frazier and Rayner 1982; Rayner and Duffy 1986). More cognitively taxing are infrequent, abstract, or ambiguous words; unconventional metaphors; long, syntactically complex, unpredictable sentences; texts lacking in coherence (Staub and Rayner 2007). Since cognitive effort is tied to specific features of the text, a ST will to some extent affect the outcome of translation process (the TT), and the TT – when used as an experimental text in a reading study – will modulate readers’ cognitive effort.

Insights into the reading process, advanced by studies on single-word and sentence processing, show cognitive effort to be affected by word- and text length (e.g., Cop et al. 2015). In empirical attempts to tie cognitive effort in translation production with translation reception, measures independent of text length may be more accurate to consider, as they allow for between-studies comparisons of the invested processing effort (see O’Brien 2010). Character-adjusted eye-movement measures have already been employed to quantify cognitive effort in entire-text reading experiments: to examine repeated exposure effects (Hyönä and Niemi 1990), text readability (O’Brien 2010), processing of foreignised elements (Kruger 2013), and translation reception (Walker 2019; Whyatt et al. 2025).

This solution makes comparisons possible between the amounts of cognitive effort invested in reading texts, despite variability in text length. In an exploratory proof-of-concept experiment marrying the translation process to reception, Whyatt et al. (2023) use character-adjusted dwell time and fixation count as proxies for cognitive effort. In studies that tie the reading process to translator’s effort, including spaces as additional characters in calculations seems justified – effortless as pressing the spacebar may appear, it contributes to the translator’s technical effort (Krings 2001). Yet, across CTIS and RS, the decision to include or exclude spaces in such adjustments varies depending on the research problem and methodological design. As a case in point, research questions that address language processing in a single language tend to exclude spaces from the total count of characters in a sentence or in a text. In such cases, there is no need to adjust for the word length that may typologically vary across the explored languages (e.g., Polish, as an inflected language, has richer morphology than English).

Another factor reported to modulate cognitive effort is text complexity. That may be assessed within the methodological framework of grammatical intricacy and lexical density (Yu and Wu 2019). Complex syntactic patterns (e.g., object relative clauses) and dense information integration in a text (e.g., many content words, uncommon lexical items) increase readers’ reaction times and gaze durations (Singh et al. 2016). While readability formulas have been reported to successfully predict the processing effort involved in translating and reading (e.g., Vieira 2014), it appears that high-complexity-but-high-quality translations might still be less cognitively taxing to read than low-complexity-but-low-quality translations (Whyatt et al. 2023, 2025). Readability metrics should therefore be employed with caution as predictors of cognitive effort in reading TTs of varying quality.

What tends to be associated with specific levels of text complexity are text types. In CTIS, some studies found text type to predict the translator’s time and processing effort invested in translation: different text types were associated with different processing efforts (e.g., Wang and Daghigh 2024; Whyatt et al. 2021). The choice of text type (e.g., descriptive, functional, literary, a news story) as experimental stimuli seems then pivotal. Expressive literary texts appear to allow for more emotional and cognitive engagement and hence might be more taxing to process, translate or read than descriptive texts. When translating literary texts, the translators may use more creative translation strategies (e.g., less frequent vocabulary or unusual grammar structures to convey the intended tone), which overall may increase cognitive effort involved in reading, yet it does not necessarily reflect the translation quality. Functional descriptive texts are assumed to be more standardised in terms of lexis than more creative texts (Tomczak and Whyatt 2022: 126) and thus potentially limited as to their effects on readers.

On the other hand, texts with language for special purposes can be rich in low frequent terms which would increase cognitive demands. In a reading experiment with excerpts that could pass as both news or literary stories, Zwaan (1994: 930) established that, when the participants believed to be reading a news story, they allocated “more resources to the construction of a causal-situation model,” and exhibited shorter reading times. Readers who believed to be reading literary stories allocated “more resources to surface-level and textbase-level processes,” which manifested in longer reading times. Minimising unwanted variability and potential confounds stemming from the choice of text allows for a more controlled exploration of the reading process, and thus raises confidence in ascribing the captured cognitive effort to the investigated factor rather than other factors.

2.1.3 The choice of experimental task to gauge cognitive effort

Task effects serve as a considerable source of variability in the processing effort (Horiba 2000). Shreve et al. (1993) found that anticipation of translation problems involved in reading-for-translation somewhat slowed down the reading process, as compared to careful reading-for-comprehension. An eyetracking study by Schaeffer et al. (2017) revealed that the total reading time, and the number of fixations and regressions approximately doubled while reading for translation, when compared to reading for comprehension. The authors argue that this lends support to the assumption that the reading purpose and task instructions have effects both on lexical access and meaning integration processes. Hvelplund (2017) examined the distribution of cognitive effort across four types of reading during translation: ST reading, ST reading while typing, TT reading while it is emerging, and then when it is complete. Reading of the emerging TT was found to be most cognitively demanding (longer fixations), followed by reading the existing TT, ST reading (without typing), and ST reading while typing which attracted shortest fixations.

Ingrained in the translation task, translation directionality has often been shown to modulate cognitive effort of translators. Translating into L2 tends to be more demanding than translating into L1 (Buchweitz and Alves 2006; Ferreira et al. 2016) and invites more extensive use of online resources (Kuznik and Olalla-Soler 2018; Whyatt et al. 2021). In reading studies, a considerable portion of variability in data can be attributed to the choice of language (here, readers’ L1 or L2) for the reading task and the purpose of reading. As evidenced by psycho- and neurolinguistic research, reading in L2 is more cognitively taxing than reading in L1. It typically involves longer reading times than L1 reading (e.g., Cop et al. 2015; Nahatame 2023).

Studying the effects of task on cognitive effort presents challenges, as it remains difficult to isolate them from other cognitive processes that become co-activated during task performance. To tease apart entangled cognitive processes involved in the translation task or reading task, and limit the extraneous factors to ensure valid and reliable measurement of cognitive effort, methodological rigour remains a priority (Fleming et al. 2023: 290). On the other hand, highly controlled studies carried out in experimental settings are, by definition, often limited in their ecological validity. Thus, reconciling high control of potential confounding variables and high ecological validity remains an ever-green challenge in itself.

3.3.1 Instruments used to profile the participants

The participants completed the Language History Questionnaire (LHQ 3.0, Li et al. 2020), used to gauge their self-reported language proficiency in L1 (Polish) and L2 (English – the language of the source text in the translation-process-and-product study), the number of years of L2 (English) use, age of L2 acquisition, language dominance, language background, exposure and use. They also took the LexTALE test (Lemhöfer and Broersma 2012) that provided more information about their L2 (English) vocabulary knowledge and filled out a survey about their reading habits, which both helped to provide more information about their profiles.

3.3.2 Experimental texts

The data recorded for two texts of different translation quality (high vs. low) were submitted to statistical analyses to answer the two primary research questions. Both texts were Polish translations of English product descriptions (L2→L1) by professional bidirectional translators in the EDiT project (see Tomczak and Whyatt 2022; Whyatt 2018b). A descriptive text type (describing a ceiling fan) was selected for further analysis for its lower potential to evoke emotional engagement in reading, which could otherwise bias the investigated relationship between the translator’s effort, translation quality, and reader’s cognitive effort.

The TT requiring extensive correction by proofreaders (17 corrections in total) was classified as low-quality (LQ); the counterpart TT requiring minimal correction for publication (only 2 minor errors) was classified as high-quality (HQ) – details on proofreader corrections and quality assessment in Whyatt (2019: 86) and Whyatt et al. (2023). Two uncorrected TTs were used in the reading experiment. They differed in terms of word- and character length (including spaces): LQ TT (1011 characters, 136 words, 8 sentences), HQ TT (1138 characters, 145 words, 8 sentences), as compared to the ST (941 characters, 162 words, 8 sentences). The readability measures (jasnopis.pl) reveal that both translations are difficult to read, with a higher index of difficulty for the HQ TT (the text-level FOG index = 19.11) than for the LQ TT (the text-level FOG index = 13.86), both more difficult to read than the ST in English (the text-level FOG index = 12.57).

The two TTs (English→Polish) of different quality were end products by two professional bidirectional translators. Demographics and data records of the translation process collected in our previous project (EDiT using keylogging, eyetracking, screen capture) show that the LQ translator had fewer years of professional experience than the HQ translator (3 years vs. 25 years), lower proficiency in English (LexTALE score = 71.25 vs. 91.25), allocated considerably less time to revision (20 s vs. 479 s) and drafting (578 s vs. 830 s), but comparable time as the HQ translator to orientation (50 s vs. 57 s). Each translator produced one TT, in the next stage evaluated by the two proofreaders and one professional translator (see Table 1 in §3.1) as a TT of LQ or HQ. Each TT contained 8 sentences, reflecting the structure of the 8-sentence ST. The translator’s effort was recorded using keylogging (Translog II) and factored in and analysed at sentence level, with varying amounts of effort put into translating each sentence. As a result, the LQ TT contained 8 sentences into which the less experienced translator put varied effort (including low and high). In the same vein, the HQ TT contained 8 sentences into which the more experienced translator put varied effort (including low and high).

3.6.1 Significant interaction effect of translator's effort and translation quality on TT readers' dwell time

Linear mixed-effects (LME) analyses tested the statistical significance of the investigated effects on dwell time and yielded no significant effect of translator’s effort nor of translation quality (b = −0.006, SE = 0.005, t = −1.199, p = 0.231; b = −3.837, SE = 2.782, t = −1.379, p = 0.172, respectively). However, a significant interaction effect of translator’s effort and translation quality on the reader’s dwell time (b = 0.022, SE = 0.010, t = 2.261, p = 0.024) was revealed, indicating that the way translator’s effort and translation quality affect reader’s dwell time is more complex. Interestingly, the examined simple effect of translator’s effort on the reader’s dwell time reached significance only for reading the high-quality translation (b = 0.005, SE = 0.002, t = 2.147, p = 0.032), and not for reading the LQ TT (p = 0.075). What follows, in the TT which was evaluated by proofreaders as high-quality, those sentences into which translator put more effort, showed higher reading dwell time among the readers (b = 0.005, SE = 0.002, t = 2.147, p = 0.032). Moreover, the readers’ dwell time was higher for reading the LQ than HQ TT (LQ > HQ) only for those sentences in the TTs into which the translators put low effort (b = −10.863, SE = 3.555, t = −3.056, p = 0.003). Neither L2 proficiency (p = 0.825) nor years of L2 use (p = 0.109) reached significance in the tested LME model (see Figure 1).

Figure 1:

Plots for the simple slope analyses for the significant interaction effect of the translator’s effort and translation quality on the reader’s effort (indexed by eye-movement dwell time).

3.6.2 Significant interaction effect of translator's effort and translation quality on TT readers' number of runs and re-reading dwell time

Two eye-movement measures taken to reflect late-stage cognitive processing (meaning integration) were examined: number of runs in the text and re-reading dwell time. The LME analyses performed for the number of runs are reported first. Significant effects were found for the invested translator’s effort (b = −0.001, SE = 0.0001, t = −2.423, p = 0.016), and for its interaction with translation quality (b = 0.002, SE = 0.001, t = 2.563, p = 0.011). In general, when the translator’s effort invested in translation was low, the readers re-visited the TT more frequently (i.e., showed a higher number of runs). However, this general finding is driven mainly by one level of the independent variable: LQ TT (b = −0.002, SE = 0.001, t = −2.573, p = 0.010), not HQ (p = 0.797). The difference in the readers’ number of runs to sentences (AOIs) in the HQ and LQ text was significant only where the translators put less effort to translate the corresponding sentences (b = −0.523, SE = 0.256, t = −2.041, p = 0.043), with more runs in the LQ TT (LQ > HQ).

The tested effect of L2 proficiency on the number of runs did not reach statistical significance (p = 0.508). However, the effect of years of L2 use was significant (b = −0.093, SE = 0.045, t = −2.055, p = 0.044). The higher number of years that the readers had been using English turned out to be linked to the lower cognitive effort they exerted to read the text that was a translation from English into Polish (see Figure 2).

Figure 2:

Plots for the simple slope analyses for the significant interaction effect of the translator’s effort and translation quality on the reader’s effort (indexed by number of runs).

Next, re-reading dwell time was examined. The LME analyses yielded a significant effect of translator’s effort on re-reading dwell time (b = −1.979, SE = 0.774, t = −2.557, p = 0.011), as well as a significant effect of the number of years of L2 use (b = −226.459, SE = 81.092, t = −2.793, p = 0.007). Overall, when translator’s effort increases, the reader’s effort decreases. Likewise, when the number of years of L2 use increases, re-reading dwell time decreases. The tested interaction effect of translator’s effort and translation quality reached statistical significance (b = 5.117, SE = 1.548, t = 3.306, p = 0.001). As in the case of the number of runs, the simple effect of translator’s effort is only significant for re-reading LQ TT (b = −4.537, SE = 1.500, t = −3.026, p = 0.003), not HQ TT (p = 0.131). When the LQ translator put more effort into translating sentences into L1, the readers spent less time re-reading those sentences.

Moreover, in the case of low effort invested by the translator into rendering sentences, the readers found it more cognitively taxing to read those in the LQ TT than in the HQ TT (LQ > HQ; b = −1593.927, SE = 512.086, t = −3.113, p = 0.002). When the translator’s effort was high, readers showed lower re-reading dwell time (indexing lower cognitive effort) in the LQ than HQ TT (LQ < HQ) (b = 1643.926, SE = 706.102, t = 2.328, p = 0.020). Neither translation quality (p = 0.947) nor L2 proficiency (p = 0.697) reached significance in the tested LME model, indicating no significant impact of the two on participants’ re-reading dwell time (see Figure 3).

Figure 3:

Plots for the simple slope analyses for the significant interaction effect of the translator’s effort and translation quality on the reader’s effort (indexed by re-reading dwell time).

3.6.3 Testing the moderated mediation model

We further investigated whether there was a relationship between the number of years of L2 use, L2 proficiency, and dwell time (more global eyetracking measure), and two late-stage processing eyetracking measures taken as proxies for cognitive effort involved in re-processing, re-analysis, and meaning integration: the number of runs (passes), and re-reading dwell time.

The moderated mediation analysis reveals that, in the tested mediation model (years of L2 use – L2 proficiency – dwell time), translation quality moderates the relationship between L2 proficiency and dwell time (b = 67.796, 95 % CI [10.039; 125.553], β = 2.585, p = 0.021). The indirect effect for the mediation model is significant for the LQ TT (b = −0.736, 95 % CI [−1.367; −0.106], β = −0.139, p = 0.022), but not for the HQ TT (p = 0.227).

When the participants read the LQ TT, their L2 proficiency served as a mediator in the relationship between their years of L2 use and their dwell time invested in reading the text. In this relationship, their number of years of L2 use was positively related to their L2 proficiency (b = 0.015, 95 % CI [0.006; 0.025], β = 0.371, p = 0.002), whereas their L2 proficiency was negatively related to their dwell time (b = −48.864, 95 % CI [−77.137; −20.591], β = −0.376, p < 0.001). See Figure 4.

Figure 4:

L2 proficiency as a mediator in the relationship between years of L2 use and (total) dwell time for reading the low-quality and high-quality translation (TT) – a moderated mediation model.

The moderated mediation analysis reveals that in the mediation model (years of L2 use – L2 proficiency – number of runs) translation quality moderates the relationship between years of L2 use and number of runs (b = −1.552, 95 % CI [−2.796; −0.308], β = −0.296, p = 0.015). Again, the indirect effect for the tested mediation model emerges as significant only for the LQ TT (b = −0.301, 95 % CI [−0.596; −0.006], β = −0.112, p = 0.046). When the participants read the LQ TT, their L2 proficiency mediates the relationship between their number of years they use L2 and their number of runs they produced when reading the text. Moreover, the number of years of L2 use is positively related to L2 proficiency (b = 0.015, 95 % CI [0.006; 0.025], β = 0.371, p = 0.002), whereas their L2 proficiency is negatively related to the number of runs (b = −19.963, 95 % CI [−34.949; −4.976], β = −0.301, p = 0.009). In the case of the HQ TT, the investigated indirect effect was not statistically significant (p = 0.912). See Figure 5.

Figure 5:

L2 proficiency as a mediator in the relationship between years of L2 use and numbers of runs for reading the low-quality and high-quality translation (TT) – a moderated mediation model.

To further corroborate the answer to the research question.

Does the mediation relationship L2 years of use – L2 proficiency – late-stage processing measures differ depending on translation quality?

In the third step, the moderated mediation analysis for re-reading dwell time was performed. In line with the findings for the number of runs, the analysis reveals that in the mediation model (years of L2 use – L2 proficiency – re-reading dwell time), translation quality moderates the relationship between the number of years of L2 use and re-reading dwell time (b = −2.553, 95% CI [−4.822; −0.284], β = −0.259, p = 0.027). Likewise, the observed indirect effect for the mediation relationship is significant for reading the LQ TT only (b = −0.647, 95% CI [−1.227; −0.067], β = −0.126, p = 0.029), not for the HQ TT (p = 0.983). In this model, the number of participants’ years of L2 use was positively related to their L2 proficiency (b = 0.015, 95% CI [0.006; 0.025], β = 0.371, p = 0.002). Their L2 proficiency, on the other hand, was negatively related to their dwell time they invested in re-reading the text (b = −42.943, 95% CI [−70.270; −15.615], β = −0.340, p = 0.002).