A bibliometric analysis of studies on multimodal translation: retrospect and prospect

3.1 Data collection

Data in this study were retrieved from the Web of Science (WoS; https://webofknowledge.com/), a publisher-independent global citation database with a search engine that helps navigate the required dataset with complete citation details (Haq et al. 2021).

To avoid bias from frequent database renewal, all the literature retrieval and data downloading were conducted in the same day, i.e., January 12, 2024. And there was no restriction on language and document type in order to collect as many documents about MT as possible. Timespan box was set to All Years (1900–2023) by default. The query string used in the advanced search of WoS for all documents regarding MT is “TS = (multimodal*/multi-modal* AND translation).”^[1] The research fields apart from arts and humanities are manually screened out, resulting in 276 publications.

3.2 Bibliometric analysis

Bibliometric analysis is a quantitative method combining mathematics and statistics, focusing on the bibliometric characteristics of research in a particular field. It helps investigators grasp the development priorities and trends in the field and guides their follow-up work (Ellegaard and Wallin 2015). With the advent of the big data era, the scale of knowledge storage and processing is becoming larger, and bibliometric analysis has become an important way to visualize knowledge relationships. Bibliometric analysis has expanded its focus on topics, publications, countries, authors, institutions, and journals in many research fields (Zyoud et al. 2019) like linguistics (e.g., Lei et al. 2023; Wang and Ping 2023), literature (e.g., Domínguez 2021; Sleimen et al. 2016), chemistry (e.g., Kusuma and Jaya 2023), computer science (e.g., Khurana and Sharma 2023; Zeini et al. 2023), etc.

However, bibliometrics has its limitations. First, the data are mainly from journals, while other document types like newspapers and monographs cannot be covered (Yang 2021). Second, there is a lag in the publication of research papers, creating difficulties in studying the current focus of a discipline (Zhang et al. 2008). Third, analyzing the hot issues solely from bibliometric indicators such as the number of publications, authors, and keywords may result in one-sidedness of the analysis results (Zhang et al. 2008).

VOSviewer software (version 1.6.16; https://www.vosviewer.com/) is used to visualize bibliometric networks, in which color, circle size, and thickness of connecting lines are used to represent units belonging to one cluster or group, size of productivity or citations, and relative link (collaboration) strength, respectively (van Eck and Waltman 2020).

4.1 Distribution of publications

Through searching on the retrieval system of WoS database, 276 publications on MT were collected, including 206 journal articles, 24 book chapters, 25 book reviews, 17 proceedings, and 4 editorial materials. As can be seen from Figure 1, the line chart intuitively presents the overall trends of MT from a slow start to rapid development. In general, there is an increase in the number of publications with a slight drop in the middle.

Figure 1:

The publication trends of MT (2007–2023).

Three stages can be summarized according to the timeline of Figure 1, i.e., a preliminary germination stage (2007–2012), a slow rising stage (2013–2017), and a rapid growth stage (2018–2023). MT has not received enough attention until 2007. Most of the TS concentrate on the linguistic dimension and rarely discuss the process, properties, and methods of generating and translating non-linguistic meaning.

In the context of globalization, the scope of translation has been largely extended and the media through which translation activities travel have undergone great changes. Though at the first stage (2007–2012), AVT (e.g., Braun 2011; Mubenga 2009; Pettit 2007) and advertising translation (e.g., De Pedro Ricoy 2007; Smith 2008; Torresi 2008) shape the mainstream of MT, the embryonic development has begun.

With regard to the second stage (2013–2017), Kaindl’s (2013) detailed grouping of translation based on modes, media, and genres is helpful for further development of MT, affecting many scholars (e.g., Aru 2013; Bernardini et al. 2016; Segala and de Quadros 2015; Slapkauskaite 2014). In addition, Munday (2014) points out that advertising translation needs more study from a multimodal perspective; Ketola (2016) builds a framework for the cognitive dimension of MT; Pérez-González (2014a, 2014b) advocates transcending modal semiotic features and integrating multimodality into translation and interpretation studies; van Doorslaer and Raw (2016) conducts a dialogue between TS and adaptation studies, aiming to deal with the question of parallels, common grounds, differences, and boundaries between the two disciplines. Actually, there are many more academic contributions during this period and all these attempts provide opportunities for the accelerated development of MT.

In the third stage (2018–2023), the subject of MT becomes more diverse, leading to the increase in publications though with a slight fluctuation. MT, interdisciplinary in nature, has gradually expanded with new perspectives such as psychology and sociology in recent years. Furthermore, electronic discourse in the digital age constructs the context of hypertext, enabling multimodality with diversified semiotic systems (e.g., the translation of websites, software, and games, in which the text, image, audio, video, and other modes are embedded; Wu 2021). For example, Theodoropoulou (2016) discusses some excerpted pictures and texts from the Aganaktismenoi pages on Facebook; Zhang and Cassany (2019) exemplifies relevant characteristics of digital communities; Yang (2021) analyzes the danmu interface overlaid onto the video; Wan (2022) discusses the multimodal interpretations of Chinese cultural terminology; Chen (2023) investigates the translation of multimodal public notices in Macao from the perspective of visual social semiotics.

4.2 Distribution of research fields

The distribution of research fields of the 276 publications is shown in Table 1. The discipline outside the brackets represents its overarching research field, while those within indicate its sub-branches or interdisciplinary subjects.

Based on the list, articles under “Linguistics” are the most common (173, 62.7 %), occupying an absolute advantage. TS initially remained a purely language-centered discipline that opened up to text linguistics in the 1970s. Then Roman Jakobson’s (1959) statement that we communicate not only with language but also other communicative means to make ourselves understood, found resonance in TS. Other popular research fields of MT include literature, communication, and education. Literature, which Tomalin (2020) describes as an imperfect umbrella term for a wide range of distinctive forms of communication (e.g., novels, short stories, sermons, essays, sonnets, epic poems), is naturally multimodal. This multimodal nature will be accentuated especially when general technological advances have altered writing traditions. For example, María Mencía’s (2017) El Winnipeg: El barco de la Esperanza is an interactive work of online e-literature combining images and texts. Tian et al. (2021) proposes a theoretical model for recontextualizing ancient classics by discussing Gopal Sukhu’s translation of the Chuci where various paratexts were added to help readers better understand. Chen (2018) analyzes the Chinese classic Mulanci through verbal-visual translation. Lee (2013) discusses a multimodal perspective on translation and a translational perspective on multimodal expression in this digital age based on literary projects. Also benefiting from the advancement of technology, studies on MT in areas of communication (e.g., Aboitiz 2018; Altahmazi 2020; Huang and Archer 2014; Lu 2023; Yang 2021), and education (e.g., Albres 2015; Beseghi 2013; Borsos 2019; Prieto-Velasco and Fuentes-Luque 2016) are sought-after.

Another notable point of MT is the emergence of new research fields like psychology, film, radio, and television (especially in AVT and subtitle translation), business and economics, history, music, religion, etc. Specifically, Wei (2016) studies the emotional factors of the translator in translation teaching, translator training, machine translation, etc., from the perspective of psychology and cognitive science; Jing (2021) examines the interaction between subtitles and other semiotic resources in films within a systemic functional semiotic framework; Ungureanu (2023) investigates how the setup of science and technology parks mediates the practices of actors in the translation chain; Spadaro (2020) proposes an original framework to understand the power of comics as a transcultural medium by exploring comics visualizing histories of migration and translation in Italy and the Mediterranean; Virc (2021) explores the translation of the aria “Tacea la notte placida” and presents several new translation improvements; Winedt (2021) addresses the need to work out the incarnational aspect of Bible translation in terms of a conscious multimodal perspective, proving that multimodal theory can offer a valuable tool to regard all types and modes of translation as equal. Nonetheless, there are still very few research articles on MT in these areas, fostering further study.

From the above, it can be seen that MT may present in many fields flexibly, in Pinto and Adami’s (2020) words, cross disciplinary boundaries necessarily. They list four interdisciplinary directions: (1) studies on relevance, narratology salience, and cognitive load and processing, as well as a cultural mapping and comparative expansion of extant cognitive approaches to multimodality; (2) specialized fields such as visual communication, media studies, graphic design, musicology, art history, and film studies; (3) studies on culture in anthropology, sociology and intercultural communication; (4) disciplines such as psychology, sensory ethnography, material semiotics and cognitive neuroscience. Our findings here corroborate their view. However, some directions they mentioned may not be highlighted in Table 1, such as anthropology and sensory ethnography; this may be due to the limitation of WoS database.

4.3 Distribution of research countries/regions

According to statistics, 276 publications on MT come from 44 countries/regions. Co-authorship analysis reveals the network of its main research countries (see Figure 2), in which a node represents a specific country. A larger node indicates the greater influence of the country; the distance and thickness of links between them indicate cooperative relationships (Zhang et al. 2021). It is obvious that many countries and regions have carried out academic cooperation and exchanges in the field of MT. And several large nodes like where China and Spain are located shows their great impact on this field. In addition, the frequency of academic exchanges between countries or regions may relate to geographical and political factors.

Figure 2:

Network of main research countries/regions in MT.

The top 10 countries/regions in the ranking of publications in MT are listed in Table 2. The results show that China (34, 12.3 %) holds a dominant position among all countries/regions. It is also noticeable that the gap between the first three countries in research productivity is very small, with China maintaining a slender lead over Spain (33, 11.95 %) and England (32, 11.59 %). Interestingly, China, as a developing country, has surpassed traditionally developed publication powerhouses with a strong publishing volume, proving that developing countries (e.g., Brazil, South Africa, Poland) have begun to play an important role in this field. This validates observations by Huan and Guan’s (2020) bibliometric study of discourse analysis (1978–2018), in which they found that China’s output has experienced a 19.8 % upsurge in the past four decades and successfully ranked fourth in the number of documents issued. Likewise, Lei and Liu (2019) also reached a conclusion in their bibliometric analysis of applied linguistics (2005–2016) that China (including Hong Kong and Macau) took third place on the list of most productive countries/regions between 2013 and 2016.

Several factors may contribute to this trend. Firstly, the financial support from the government encourages researchers, especially in China, where government funding for research has been increasing annually (Lei and Liu 2019; Zhou et al. 2009). The second is perhaps relevant to the rising soft power of China in knowledge production (Huan and Guan 2020). And the third contributory element may turn out to be the strong export of knowledge from traditional powerhouse countries, as MT emerged and developed in western countries much earlier, leading both theoretically and practically. However, existing research has problems such as having a single research perspective, a narrow research field, and unsystematic analysis (Xu 2017), motivating researchers to catch up and explore new fields.

4.4 Distribution of productive authors

The authors who published at least two articles in MT are shown in Figure 3, with 40 of all 395 authors meeting the above requirement. We can still find that a large proportion of prolific authors publish their articles separately, generally consistent with the dominant status of single-authorship in the whole TS community (Cronin 2003; Huang and Xin 2020).

Figure 3:

Network of main productive authors in MT.

As shown in Table 3, the 10 most productive authors contributed 36 articles, accounting for around 13.04 % of the total MT publications until 2023. Most of the authors study multimodality in translation, including well-known researchers in AVT or subtitle translation (e.g., Braun 2011, 2016; Guillot 2020; Jiménez Hurtado and Soler Gallego 2013; Pinto 2018) and picturebooks (Chen 2022). Besides, researchers focusing on pragmatics (Dicerto 2018) and neurolinguistics (Nunez 2018) also pay attention to MT. For example, Dicerto (2018) has been working on pragmatics and multimodal studies in translation for many years with her monograph Multimodal Pragmatics and Translation, providing a model for its application. Nunez (2018) reviews the methodological contribution that social neuroscience could make to the study of MT, and finally presents an experimental design emphasizing the transfer of feelings and emotions in film audio description. Thus, interdisciplinarity provides new insights for MT.

Furthermore, most of the articles with high citations are theoretical studies. Specifically, Jiménez Hurtado and Soler Gallego (2013) investigate the relationship between multimodality, translation, and accessibility based on a corpus of audio descriptions. Braun (2016) reviews some cognitive and pragmatic models and addresses their contributions to AVT. Tuominen et al. (2018) initiates a discussion on how to engage in MT to clarify some basic definitions, rationales, etc., providing a good reference.

4.5 Distribution of source journals

Statistics show that the 276 publications about MT are widely distributed across 147 different journals. The minimum number of publications is set to two, and a total of 37 journals meet the threshold (see Figure 4). A node represents a specific journal, with the size of nodes indicating the influence of the journal. Connections between nodes showcase the cooperative relationship of different journals. Notably, Perspectives-Studies in Translation Theory and Practice and Journal of Specialized Translation gain much attention in the landscape of journals related to MT. There is a marked imbalance in the relationship and cooperation among journals. Some journals have strong cooperative ties with many other ones such as Perspectives, Meta, and Journal of Specialized Translation, while some others are isolated, like Onomazein and Frontiers in Psychology. This may because the former group of journals is on TS with more frequent connections, whereas the latter is not.

Figure 4:

Network of main source journals of publications in MT.

The top 10 productive journals in MT are listed in Table 4. The journal Perspectives, holding the highest number of publications and citations on MT, has a long established history, coupled with the thematic inclination or scope of its selection (i.e., paper submissions exploring various linguistic and cultural mediation and interdisciplinary studies are encouraged). Publications by leading linguists in a field have a lasting impact on the journal that publishes them. For example, O’Sullivan and Jeffcote edited a special issue of the Journal of Specialized Translation to discuss “Translating Multimodalities” in 2013, which may attract the attention of many researchers later, thereby increasing the number of submissions and citations of related articles to the journal. Coincidently, Jiménez Hurtado, Tuominen and Ketola also edited a volume of Linguistica Antverpiensia New Series-Themes in Translation Studies (LANS-TTS) exploring “Methods for the Study of Multimodality in Translation” five years later. And the possible reason why LANS-TTS has fewer publications and citations than the previous two may be that it is an annual journal and therefore publishes fewer articles. In addition, it is interesting that the number of publications is not directly proportional to the number of citations. For example, although the journal Meta has more citations, the number of articles published in this field is not noteworthy. This may be inseparable from the long publishing history of Meta, which was founded in 1956, while most of the other journals began publishing later.

4.6 Distribution of author keywords

Here the minimum number of occurrences of a keyword is set to two and a total of 20 author keywords are finally extracted from 276 publications, which are displayed in Table 5. As mentioned earlier, the research hotspots and important topics may be identified by analyzing high-frequency keywords, as they usually highly concentrate and summarize the core elements of the literature. Unsurprisingly, the keywords of “multimodality” and “translation” are at the top, with the most conspicuous occurrences on account of the research focus of what we are discussing. Of particular note, the keyword “audiovisual translation” (or AVT) received wide attention, reflecting the interests of many scholars in MT. Some studies have already discussed the disciplinary connections between multimodality and AVT (e.g., O’Sullivan 2011; Pérez-González 2007; Taylor 2014; Xu 2017). As mentioned before, the earliest and most widely studied field of multimodality and translation is AVT (González 2020). Consequently, some AVT-related topics (e.g., subtitling, dubbing, audio description, sight translation) are of great interest.

Additionally, high-frequency content words including “intersemiotic translation,” “social semiotics,” “adaptation,” “ST analysis,” “retranslation,” interpretive “resemblance,” “localization” and “accessibility” may specify the research approaches and strategies. To be more specific, social semiotics views translation as meaning transposition in the multimodal semiotic landscape (Kress 2020), which helps understand how different modes are used as semiotic resources to promote meaning-making. It is also proved that there is an inseparable affinity between MT and semiotics. In translation, ST analysis should be carried out, and full consideration should be given to adaptation, interpretive resemblance, localization, and accessibility. Furthermore, retranslation is required by the very nature of multimodal texts under scrutiny, such as film, music, graphic novels, etc., and might be pondered if the existing translation does not meet requirements.

Here it is worth mentioning that comics give an intriguing window into the genre of research material in which scholars are interested. For instance, Huang and Archer (2014) explore the fluidity of modes in manga by using Naruto (written by Masashi Kishimoto in 2003) as a case study. Borodo (2015) investigates how to exploit the relationship between verbal and visual modes in translation from a multimodal perspective by taking the classic, Franco-Belgian comic book series Thorgal and its Polish translations as instances. Fu (2019) puts forward comic translation strategies on the basis of relevance theory through a comparative analysis of Spanish comics and Chinese ones, also indicating the importance of relevance theory and translation theory in MT. Moreover, the presence of machine translation may point to another aspect of MT, in which interaction emphasizes its nature. Based on a two-hour interaction involving a family of Albanian asylum seekers, Piccoli (2022) deals with the use of Google Translate as one among many resources that participants mobilize to overcome the language barrier in plurilingual medical consultations. Karakanta (2022) proposes a set of recommendations towards achieving replicability and reproducibility in experimental research at the crossroads between AVT and MT. However, comics and machine translation are just the tip of the iceberg among all the studies on MT. Actually, many other underrepresented material of MT can be studied such as literary works (e.g., Aboluwade 2019; Chen 2018; Lee 2013), advertisements (e.g., Pan 2015; Pardi et al. 2018), musical lyrics (e.g., Jin 2015), etc.

4.7 Analysis of highly cited articles

The minimum number of citations of an article is set to two and a total of 61 articles meet the threshold (see Figure 5). Here the node size shows the number of citations. It can be observed that Doerr (2017) has the largest node indicating the highest number of citations. This may be due to the broad interdisciplinary nature of her research topic, which appeals to scholars in fields like political science, psychology, sociology, etc., resulting in a high number of citations from a wider range of sources. In terms of the time span, most of the highly-cited articles were published in earlier years, confirming Lei and Liu’s (2019) conclusion that when other factors are held constant, a publication less than or equal to three years tends to generate substantially fewer citations than a publication with an age of five years or more. In addition, it can be clearly observed that only a few articles show correlations, while most remain irrelevant and thus appear as isolated and scattered nodes in Figure 4. That probably because most of the authors have no cooperation or the cooperative research is still in a small number.

Figure 5:

Network of highly cited articles in MT.

The list of top 10 highly cited articles in MT is presented in Table 6. The more one article is cited, the greater its influence in this field. As regards the research themes, over half (60 %) of these articles are concerned with AVT (i.e., Baumgarten 2008; Bednarek 2015; Braun 2011; Caballero 2009; Guillot 2012; Jiménez Hurtado and Soler Gallego 2013), proving again that AVT is still the mainstream of MT, and confirming its forefront position in keyword rankings. The development of science and technology will bring more innovative audiovisual materials and greater research potential. Two of the remaining four articles focus on the theoretical aspects of multimodality and translation (i.e., O’Sullivan 2013; Snell-Hornby 2009), and two on cartoon and comic translation (i.e., Borodo 2015; Doerr 2017). This result may provide evidence to the popularity of research topics such as intersemiotic translation, social semiotics, comics translation, translation theory, etc. Another finding worth mentioning is that the top 10 highly cited articles only involve four productive authors (i.e., Jiménez Hurtado, Guillot, Gallego, and Braun) listed in Table 3, suggesting that though some researchers are highly productive, the absence of a seminal work may result in a lack of broad attention. On the contrary, some scholars with relatively low output can also make a significant impact on MT due to their remarkable contributions.

4.8 Analysis of highly cited references

Out of the 5414 cited references, the minimum number of citations of a cited reference is set to five and 75 highly cited references are probed in the end (see Figure 6). It is generally believed that constantly and frequently cited research constitutes the knowledge foundation of a given field (Huan and Guan 2020). The size of node represents the influence of references, and links between denote their relationship (Deng 2023). Intuitively, Kress and van Leeuwen’s (2001) Multimodal Discourse is most highly cited and influential in MT. Applying systemic functional linguistics (SFL) and drawing on an enormous range of examples from various kinds of multimodal texts, this seminal work builds on its reputation as the first systematic and comprehensive account of visual grammar to investigate how individual multimodal elements are fused into a coherent unity.

Figure 6:

Network of highly cited references in MT.

Table 7 lists the top 10 highly cited references in terms of citations. Obviously, works by Kress, whether solo or co-authored, occupy the top three, indicating their fundamental role in the development of MT. Jakobson’s (1959) “On linguistic aspects of translation” and Halliday’s (1978) Language as Social Semiotic follow closely in citations and are highly influential in this field. Concerning research topics of these references, most of them are theoretical studies of MT based on SFL and social semiotics. For example, Halliday (1978) interprets the nature and meaning of language from a social and cultural perspective; van Leeuwen (2005) explains the application of multimodal research methods across texts. In this sense, these theoretical studies provide a methodological basis for MT, contributing to their high citations. In terms of publication type, eight of the top 10 are books (i.e., Thibault and Baldry 2006; Dicerto 2018; Gutt 2000; Halliday 1978; Kress 2010; Kress and van Leeuwen 1996, 2001, 2006), exactly supporting Rovira-Esteva and Orero’s (2011) conclusion that books are read and cited more often than other publication types. The possible reason may be pertinent to the higher stability of books and the time-sensitivity of journal articles.