English as a lingua franca in academic publishing: using round-trip translation to estimate linguistic revision difficulty

2.1 English as a lingua franca and academic writing

In the era of (post-)globalization linguae francae have developed as fundamental means for intercultural communication and knowledge sharing (Hall 2013). The massive spread of the English language in particular has raised many questions concerning its usage in all the different domains of human interaction (Jenkins 2013, 2015; Kachru 1992; Schneider 2012).

From a functionalist perspective (Ventola 1991), English as a lingua franca (ELF) can be defined as “any use of English among speakers of different first languages for whom English is the communicative medium of choice and often the only option” (Seidlhofer 2011: 7). Communicative behaviours, practices and interaction in ELF are continuously set and reset by users in the process of affirming their role as active speakers in different sociopolitical communicative contexts (McKinley and Rose 2018), this last aspect being crucial for a complete understanding of ELF characteristics and use (Lillis and Curry 2010).

This paper looks at ELF in connection with English for Academic Purposes (EAP) practices, defined by Hyland and Shaw as “language research and instruction that focuses on the communicative needs and practices of individuals working in academic contexts” (Hyland and Shaw 2016: 1).

Studies about ELF and EAP have frequently overlapped, both sharing a functionalist and social perspective on the use of English for transcultural communication (Bennett 2013; Hood 2016; Thompson and Diani 2015). These studies have problematized the traditional dichotomy between native and non-native speaker varieties, providing a decolonized perspective on writing and revision (Canagarajah 2002, 2023a). While the concept of nativeness is often – wrongly – perceived as directly related to proficiency in writing, research has generally acknowledged that skills in English academic writing must be acquired by both native and non-native writers (Bennett 2009; Römer 2009). Römer provides some valuable evidence in this direction, arguing that EAP is a rather specific case of non-native linguistic production, because “native speakers also have to learn […] academic writing,” adding that “[t]he native academic writer does not seem to exist” (Römer 2009: 99).

In this respect, it should be noticed that the dichotomy between native and non-native speakers, far from representing the complex plurilingual and intersemiotic reality of communication (see, for example, ‘translanguaging’ theory by García and Wei [2014]), has conventionally been tied to monolingual ideologies which encourage cultural hierarchies and linguistic hegemony at different levels (Baird et al. 2014; Larsen-Freeman 2018; Seidlhofer 2011). Academic English is one clear example of a sociolect (Canagarajah 2023b) characterized by dominant native-speaker standards which have – at least until relatively recently – flattened the plurality of voices and standpoints in international scientific communities (Amano et al. 2023). Recently, however, scholars engaged with decolonizing practices in the use of English language(s) have proposed approaches and practical examples which subvert the standards of academic English by introducing unconventional language and mixing a plurality of codes and varieties in writing, maintaining their clarity, effectiveness and adequacy to the communicative context (Canagarajah 2023b). In line with such practices, our study proposes a methodology for estimating the need for linguistic revision based on objective factors related to functional communication, rather than ideological standards unveiling an imperial representation of native speakers as the only owners of English (Widdowson 1994), as will be discussed in the next section.

2.2 Language evaluation in the academic publishing industry and English linguistic revision

Against the background of research on academic text evaluation, especially focused on learners’ assessment (Madnani and Cahill 2018; Yannakoudakis 2013), the studies on language standards and evaluation in scientific publication have mainly been developed by a niche of scholars who actively engage with the topic from the writer’s perspective, as briefly mentioned at the end of Section 2.1.

In this landscape, studies on linguistic revision and the role of revisors are still underdeveloped. Consequently, this practice has been hardly defined and terminological ambiguity still characterizes the use of expressions such as ‘revision’, ‘editing’, and ‘copy-editing’ (Wang 2021).

For the purposes of this study, we broadly consider revision as the process of applying structural, lexical or grammatical changes to a text written and/or translated by somebody else to improve its adequacy and communicative effectiveness. It should also be noted that the borders between writing in ELF and translating into ELF are blurred, as many academic writers frequently engage with self-translation and automatic translation (O’Brien et al. 2018). Indeed, literature on linguistic revision is a collage of studies on writing, translation and editing for the publishing industry (Wang 2021), which overall show a functionalist and target-oriented approach. However, most studies focusing on the evaluation of English academic writing often underestimate, and sometimes completely forget, that such evaluation has a very practical aim: instructing the practice of linguistic revision.

When investigating revision difficulty estimation (i.e., linguistic evaluation for revision purposes), the following aspects need to be considered: the revisor’s role in the publishing industry, types of revision and improvement techniques, and criteria for evaluation. The academic publishing companies’ panorama is diverse, and the organization of the editing process changes from company to company, depending on the publisher’s size and structural organization, on where the company is located and on the authority accorded to editorial boards (McGuigan and Russel 2008).

For what concerns the revisor’s role in the publishing industry, it has been noted that revision is a heavily under-recognized practice. In the internal organization of the publishing houses it is very often a process to which not much attention nor a large budget are dedicated. In fact, in most cases, revisions are made by external experts, whose compensations are estimated based on the limited available budget (Mossop 2001). For example, Willey and Tanimoto (2013) provide insights on the working situation of English teachers as unexperienced revisors, who offer more convenient rates compared to those of professionals. Their study sheds light on the necessity for specialized training to overcome the nativeness bias on revisors’ proficiency.^[2]

The degree of revision (Mossop 2001), and, more generally, the choice of one approach or another do not depend only on the revisor’s evaluation over the manuscript. Budget, time, requirements of the publishing house and target audience all play a role in the revisor’s decision process. However, manuscript evaluation is still the starting point and essential step for any type of revision.

In recent years, the concept of readability has colonized the debate on text quality evaluation (Todirascu et al. 2016). In this respect, Mossop differentiates between clarity and readability, linking clarity to meaning and readability to the formal features of a text. But readability is a rather vague criterion, which leaves space for subjective and potentially biased evaluation.

Different views regarding ELF and EAP directly affect the approach to the revision task. Hartse and Kubota (2014) distinguish between error-based and variation-based approaches to revision, the latter including World Englishes, translingual and constructionist approaches. They discuss difficulties and issues of the revision process, noticing that “editing decisions reflect a complex interplay among native-speaker intuition, desire to avoid stigmatization, and conformity to standard varieties of English” (Hartse and Kubota 2014: 80).

The authors also note that there are very few academic journals which explicitly accept articles written in English varieties that diverge from Standard Native Speaker (NS) Englishes. They conclude that the “copyeditor is the final authority on appropriate language use” (Hartse and Kubota 2014: 79). Recently, some journals, especially in the field of linguistics and applied linguistics, such as the Journal of English as a Lingua Franca, published by DeGruyter, and the ESP Journal and the Journal of Pragmatics, published by Elsevier, do not strictly enforce native-like English proficiency and instead focus on clarity, coherence, and contribution to knowledge. These journals recognize the global nature of academic discourse and encourage submissions from non-native English speakers without requiring heavy linguistic editing, sometimes also asking for bilingual/plurilingual submissions. Still, linguistic revision by native speakers is required by many journals,^[3] within and outside the field of linguistics, confirming that the issue of over-revision is still central and needs to be properly addressed (Billingham 2002; Tribble 2006). Revisors should critically think about the reasons for applying changes, to preserve “a greater variety in expression and more equity for writers”, which in turn could result in “higher linguistic and ethical standard[s]” (Hartse and Kubota 2014: 81).

To fulfil this aim, nativeness should be replaced by some more purposeful concepts related to communicative success (Gosden 1995). For a successful communication, the language used in academia should account for “the regulating mechanisms of [academic] discourse community” (Gosden 1995: 38). In this sense, Mossop exploits the term discourse expertise to refer to both terminological precision and alignment with genre conventions (2001). However, while terminological correctness is easier to evaluate, genre conventions are highly variable and socially negotiated (Canagarajah 2022; Seidlhofer 2009). From a geopolitical point of view, Canagarajah affirms that “the representation of any research […] is considerably mediated by the rhetorical processes of writing and publishing” (Canagarajah 2002: 20). Adopting a similar view, Hyland and Shaw describe the results of globalization processes in terms of “inequality in the production and distribution of knowledge” (Hyland and Shaw 2016: 95).

In conclusion, the spread of English in academia and in the academic publishing industry means that most scientific texts are written, translated or self-translated in English by L2 users, for an L2 public. At the same time, a correlation between written English proficiency and research productivity has been noticed by Vasconcelos et al. (2007), making revision an essential practice for equal access to knowledge production.

Many scholars have proposed inclusive approaches to academic writing and revision. At the same time, some considerations on language standards are necessary because academic writing, for its own nature, needs to be comprehensible and appropriate to the meaning it aims to convey (as recognized also in studies which promote inclusive plural linguistic practices in academia, such as Balida et al. [2022] and Peters and Anderson [2021]). Therefore clarity, unambiguity and readability must be assured.

The complexity of the institutional and situational contexts points to the need for more structured linguistic services in the academic publishing industry which should be aimed at creating bridges for scientific discussion rather than limits for publication. And while revision can serve as a means of empowerment and legitimation for authors whose work may otherwise be rejected simply because of language-related evaluation, inclusive views on languages, like translanguaging theories, should be encouraged within the academic publishing landscape as well as in translators and revisors training curricula, to enhance the use of non-standard varieties of English and other languages in academic communication (Canagarajah 2013; Schnell 2024).

2.3 Automatic text production evaluation, and automatic translation

In the context of automatic text processing, many studies have focused on quality evaluation of human linguistic productions. Probably the first approach to this topic as a stand-alone issue was related to the possibility of automatically assessing English learners’ productions (Cotos 2009; McNamara and Graesser 2012; Yannakoudakis 2013). In general, learners’ productions and academic writing in ELF share some features, but there is an important difference which makes it impossible to apply the same assessing methodologies on these texts, as highlighted in Section 2.2. In the publishing context, where the purpose of evaluation is very practical and target-oriented, the strongest concerns relate to linguistic standardization, simplification, and the imposition of native-like norms that may not align with authors’ communicative objectives. Technology can undoubtedly streamline revision processes and improve objectivity, yet their reliance on fixed language models can introduce biases towards standardized output (Avner et al. 2016; Vanmassenhove et al. 2021), as also suggested by our last experiment (discussed in Section 4.4).

For this reason, the applications specifically meant for revision traditionally only support the revisors in the process; for example, text editors such as Libre Office and Microsoft Word are among the most common software tools employed by revisors (Billingham 2002; Mossop 2001). There are also plenty of online text editors, such as Google Documents and the innovative Collaborative Editing Tool for Non-Native Authors (CEPT), developed by Zhu et al. (2017).

More recently some applications have been created for automatic proof-reading. A well-known example is Grammarly, which has been identified as one of the most effective intelligent editing programs available (Max 2021).^[4] Grammarly is a proprietary, cloud-based writing assistant designed for English text revision, employing a combination of grammar-based rules and machine learning techniques to provide real-time feedback. The software’s adaptive AI model refines its performance through user feedback, allowing it to improve error detection and style recommendations over time. Despite these advancements, AI-assisted proofreading tools are not neutral: they introduce implicit normative pressures, particularly in academic writing contexts, where evaluation metrics such as Grammarly’s performance score may favor native-like textual conventions (Abu et al. 2024). Privacy concerns, potential biases towards simplification, and limited adaptability to ELF features are among the critical aspects that need further scrutiny.^[5]

Automatic translation is another much-used technology in the publishing industry as it is in the academic world. Since its earliest stages (Shannon 1948; Weaver 1949), machine translation has changed a lot. State-of-the-art MT systems are based on neural networks, which make intensive use of deep learning techniques with bi-directional layers using end-to-end processes for decoding and encoding (Mikolov et al. 2010; Xue et al. 2014). The Encoder-Decoder (seq2seq) model was first used by Google in 2014; to date, it is one of the best performing neural network models for machine translation (Shah 2020). Thanks to this model, it is possible to produce outputs that are also very different from the inputs – a characteristic which is relevant to the use of MT proposed in the present work.

When it comes to MT quality evaluation (QE) two main approaches can be used: human evaluation, which is the most accurate but also expensive and time-consuming, and automatic evaluation (Rivera-Trigueros 2021). Progress in this research field is so rapid that it is unlikely that descriptions of MT architectures and performances will be found in textbooks. For the purposes of the present research, we have taken into account the Proceedings of Machine Translation Summit (Duh and Guzmán 2021) and the Intento report (2021), which were the latest available at the time the experiments were conducted. According to the report, ModernMT is one of the best-performing systems, with outstanding results in the education field.

Finally, it should be noted that the rapid advancements in large language models (LLMs) and the advent of generative AI have deeply affected text production tasks, including academic writing. AI tools such as ChatGPT have introduced new challenges for academic writing, particularly in the context of ELF and multilingual scholarship. While these AI-driven systems can assist with text structuring, coherence, and fluency, they also pose several risks in terms of over-standardization, loss of authorial agency, and potential ethical violations (Kocak 2024). Unlike the methodology proposed here, which is only meant to objectively evaluate the difficulty, hence time and effort needed, for human revision, generative AI systems work as fully automated writing assistants and their use in academic writing, although very recent, has already been attested and discussed (e.g., Bhatia 2023; Lin 2024; Zohery 2023). A comprehensive account of the issues related to the use of ChatGPT falls outside the scope of the present work. But given the centrality of such technologies in the current debate about text production, authorship, authenticity, scientific quality and linguistic standardization, a clarification is in order to highlight the differences between the methodology we propose and the assistence provided by generative AI.

First and foremost, ChatGPT and similar AI tools lack critical reasoning abilities and cannot engage in genuine scholarly argumentation (Bender et al. 2021). These models rely on statistical pattern recognition rather than an understanding of epistemic frameworks, research methodologies, or domain-specific knowledge. As a result, AI-generated text may contain plausible-sounding but incorrect, biased, or unverifiable information (Dale 2023). The problem of “hallucinated references”, where AI fabricates non-existent citations, is particularly concerning for academic integrity (Thorp 2023). This issue raises serious ethical implications, as reliance on AI-generated content without rigorous verification could contribute to misinformation in scholarly discourse.

Furthermore, unregulated AI use in academic publishing risks compromising research authenticity, as AI-generated texts may be insufficiently distinct from existing sources, leading to concerns about self-plagiarism or unintentional content recycling (van Dis et al. 2023).

From an ELF perspective, there is also the danger that AI-assisted writing may result in linguistic homogenization, where the diversity of academic English varieties is eroded in favor of standardized expressions. This trend could diminish the agency of non-native scholars, reinforcing the idea that academic credibility is linked to native-like fluency rather than intellectual contribution (Myers 2023). AI-generated texts tend to produce outputs that align with monolithic, standard English conventions, potentially marginalizing diverse linguistic expressions and reducing the visibility of non-native academic voices. As AI models are trained predominantly on existing academic and internet corpora, they may favor dominant discursive patterns while overlooking alternative rhetorical structures commonly found in multilingual scholarship (Agarwal et al. 2024; Liang et al. 2023).

Potentially, a cautious use of such technolgies – which requires a thorough understanding of their functioning and risks – could also be beneficial but the above-mentioned issues and recent studies like Xiao et al. (2025) prevent us from considering generative AI systems as safe unbiased tools for linguistic revision.

Abolishing the use of generative AI in the academic world sounds like a paradoxical proposal, but alternative paths can be explored to promote critical use of AI. Following these considerations, the methodology described below does not seek to substitute human revisors, but to provide an output which can help them understand and predict the amount of effort and time necessary to enhance communicative effectiveness of academic texts, as will be clarified in the discussion of the results (Section 4).

4.1 First sets of experiments: BLEU scores between manuscripts and revisions compared to the human evaluation of revision difficulty

The first experiments were aimed at investigating hypothesis 1, i.e., that the higher the similarity score between a manuscript and its revision, the easier the revision.

We computed BLEU scores between the manuscripts and their revisions. The scores were inverted (using the formula 1 – BLEU score) and normalized on a scale from 0 to 3 to be comparable with those deriving from the human evaluation; the resulting values are called Automatic Index of Revision Difficulty (IRev_A). The results of this experiment are shown in Table 1.

In line with the preliminary experiments, these results demonstrate that BLEU can be employed to measure the similarity between a manuscript – the candidate – and its revision – the reference. Our first hypothesis is thus generally confirmed, although the classifications provided by IRev_A and IRev_H do not overlap perfectly, as can be noticed in Figure 1.

Figure 1:

Comparison between texts ordered according to IRev_H and texts ordered according to IRev_A.

It should be noted that, in personal communication, the revisor has reviewed the task of evaluating texts according to a holistic judgement as “very difficult”. In particular, she noticed that, while texts which needed major revision interventions could be easily separated from texts which did not, it was particularly hard to assign texts to the intermediate group. It could be observed that the same difficulty is reflected in the automatic comparison: BLEU scores are more accurate with clear-cut cases. A final index of revision difficulty (IRev_F) was derived from the average between the human and the automatic indexes, which is used to interpret the results of the subsequent experiments.

4.2 Second set of experiments: BLEU scores between manuscripts and their RTT as indicators of revision difficulty

The second set of experiments aims to verify hypothesis 2, namely whether the similarity between a manuscript and its round-trip translation is indicative of the difficulty – and, therefore, costs and time – of its revision. In order to test this possibility, the scores between manuscripts and their round-trip translation were compared to the scores between the respective revisions and their round-trip translations: Rev_RevRTT is expected to be higher than Raw_RawRTT in cases of substantial revision difficulty.

More precisely, we calculated the BLEU scores between the revised texts and their RTTs. Then, we compared these scores to the scores between the manuscripts and their RTTs. In order to make the scores comparable across the dataset, we calculated the difference between Rev_RevRTT and Raw_RawRTT. This difference has been inverted and rescaled on a range 0–3 to obtain Idx_Dis. Table 2 shows the results of the experiment, and the human evaluation values for comparison in the last column (IRev_H).

If our hypothesis is correct, the discrepancy index (Idx_Dis) is a good indicator of revision difficulty (Irev_H). Despite some differences, Figure 2 shows that the discrepancy index has similar values to the benchmark indices.

Figure 2:

Comparison of discrepancy index (Idx_Dis), human revision, difficulty index (IRev_H), automatic revision difficulty index (IRev_A) and final revision difficulty index (IRev_F).

However, a closer analysis shows that there are some cases in which our hypothesis is not confirmed, namely when the manuscripts need little revision. Therefore, an ad hoc experiment on Text 47 was performed to examine the responsiveness of our method, by generating sample texts with randomly deleted words and added errors. Our results, as represented in Figure 3, show that the similarity between raw texts and their RTTs systematically decreases as text quality degrades.

Figure 3:

The BLEU scores for comparison of a sample reference text and candidates of decreasing quality. The number of randomly deleted words and/or deteriorating changes is reported on the x axis.

The results in Table 2 were then compared to Grammarly scores. The comparison, presented in Figure 4, shows an agreement between the results of the RTT-based method and the scores provided by Grammarly (the yellow bars in Figure 4). A detailed representation of three texts representing the three groups based on the revisor’s evaluation is also shown.

Figure 4:

Comparison of BLEU and grammarly scores. (a) Comparison of the Raw-RawRTT score (blue bars) with the score provided by grammarly (yellow bars). (b) Details of the results of the experiments on three sample texts, compared with the grammarly score (yellow bars).

This comparison with an external benchmark seems to confirm the validity of the RTT/BLEU method. Thereby, a third set of experiments on unrevised data was conducted.

4.3 Third set of experiments: thresholds and testing

At this point in our experimentation, BLEU values had to be kept unaltered for the identification of BLEU thresholds. Consequently, we inverted IRev_H and normalized it on a scale from 0 to 1, to make it comparable to the BLEU scores. We refer to such values as IRev_E (index of revision ease). The correspondence between BLEU values and IRev_H/ IRev_F of the texts in CIHA allowed for a classification in three groups, based on observation of the visual clustering of the scores:

1. Texts with high revision difficulties characterized by a BLEU score ≤0.6.

2. Texts with low revision difficulty characterized by a BLEU score ≥0.75.

3. Texts with intermediate revision difficulty or with unpredictable revision difficulty for which the Raw versus Raw_RTT BLEU score is between 0.6 and 0.75.

As test set, three articles from the journal New Medit published by BUP were used. They are written in English as a lingua franca by non-native speakers, as confirmed by BUP editors, and they were not revised before publication. After the segmentation procedure, the round-trip translations of each text were generated in accordance with the method described in Section 3. Table 3 shows some sample segments and their round-trip translation.

This time, since we could not rely on posteriori automatic and – most importantly – human evaluations, the results were compared only to the Grammarly scores (normalized on a scale 0–1). The results are reported in Table 4.

As the table shows, with a value lower than 0.60, the new_medit_3 text belongs with the first group. On the other hand, new_medit_ 2 and new_medit_1 have BLEU scores of 0.717 and 0.645 respectively, thus falling into the intermediate range.

Moreover, Table 4 shows that BLEU on RTT and Grammarly scores are similar/correlated, while Figure 5 shows also the classification of the documents according to the groups defined above.

Figure 5:

BLEU and grammarly scores for the New Medit test set.

These results suggest that the proposed thresholds for revision difficulty evaluation are consistent, although time and data availability limitations did not allow for further testing.

4.4 Experiments 4: evaluation of the RTT output

With the aim of better understanding MT behaviour in the context of round-trip translation, a fourth set of experiments was designed.

The similarity score between manuscript RTTs and revisions was computed and compared to the similarity between manuscripts and revisions. If the similarity between a manuscript’s RTT and its revision is higher than the similarity between a manuscript and the corresponding revision, this would mean that RTT produces an output, thus it could be used as a starting point for revision. Figure 6 shows the results of this comparison.

Figure 6:

Comparison between Raw-Rev BLEU scores and RawRTT-Rev BLEU scores.

The scores between the manuscripts (Raw) and their corresponding revisions (Rev) are systematically higher than the scores between the manuscripts round-trip translations (RawRTT) and the respective revisions (Rev). Thus, our third hypothesis has been disproved. These results indicate that the revisions by the expert revisor are closely related to the source texts: the authors’ structural and lexical choices are respected as much as possible. Table 4 provides an example extracted from text 38 of the CIHA dataset.

As can be noticed from Table 5, RTT produces random changes which are not necessary for the purposes of revision, and which – if applied by a human revisor – would be regarded as over-revision.