Volume 11, Issue 2

One of Japan’s most versatile scholars of film, television, and manga (comic books), Yomota Inuhiko (1953–) made a deep impact on Manga Studies, where he engaged in a form of manga analysis commonly called “expression theory” (hyōgen-ron). In the present translated twin essays from Yomota’s now-classic study, The Principles of Manga (Manga Genron, 1994), he demonstrates his awareness of the possibilities of applying a linguistic lens to the study of Japanese comics. In Part One, Yomota explores the idea of onomatopoeia in manga, considering how they are very much cultural constructions that can be difficult to translate into other languages. In Part Two, Yomota delves deeper into the way onomatopoeia appear on the page, often creating impossible-to-utter expressions but ones easily understood nonetheless by readers. While there is already a great deal of research on onomatopoeia mimetics in spoken Japanese, Yomota here expounds on how manga artists have invented new ways of employing onomatopoeia outside of these established patterns in the spoken language. He gives clear examples of the evolution and representation of mimetics in manga, discusses several key features of comic-book mimetics, and makes a clear argument that they have spurred the evolution of modern-day manga.

This article examines multimodal storytelling performed outside the main body text in book-length popular science. By employing a case study approach, the study examines three kinds of visually detached stories: i) fragmentary case stories of lay experience, ii) fictional stories of prehistorical lifeworld and iii) scientists’ personal stories with everyday photography. The article shows that detached stories are digressions from the locally, or also globally, dominant ways of making meaning. These digressions can be modality shifts (from verbal to multimodal) but also more complex shifts that engage with cultural boundaries of (popular) science. Furthermore, the article complements prior studies of photographic storytelling by discussing the affordances of everyday photography in public occasions. The analytic framework combines Systemic-Functional genre studies, Social Semiotic and related models of visual design as well as approaches to factuality and fictionality in Narrative Studies. The data consists of contemporary Finnish popular science books representing the natural sciences.

Universities in English speaking countries have been witnessing an increasing number of international faculty who speak English as an international language. However, the universities are mostly guided by the dominant monolingual ideologies and language policies that favor verbal repertoires in standard English even though instructional communication is characteristically multimodal. Against this backdrop, this article focuses on two faculty members in STEM and investigates their instructional interaction. Using multimodal interaction as a theoretical and methodological heuristic, the analysis pays attention to how modes, codes and material objects are ensembled in specific configurations as instructors and students engage in the process of meaning negotiation. The instructors’ instructional practices and perspectives demonstrate the importance of the entanglement of language with diverge semiotic, social and material elements of communication. The findings suggest a need for a broader definition and scope of instructional interaction in STEM.

Texts containing information in different modes are growing in number, technological sophistication, and quality presentation. In this changing semiotic landscape, it is timely to examine which mode combination(s) is most comprehensible for English learners. Accordingly, three text conditions – a verbal text, a supplementary text, and a complementary text – were examined for which mode combination(s) produces the best comprehension outcomes for English learners. Participants were intermediate learners ( n  = 111) and advanced learners ( n  = 111). Each language group was further divided into three subgroups ( n  = 37), and each of the subgroups was tasked with reading in one text condition and answering questions about the text. Participants’ comprehension scores were compared across the text conditions and language groups. The results revealed that the complementary text condition significantly impedes learners’ comprehension, and no significant difference was observed between the verbal text and the supplementary text conditions. These results indicate that learners did not engage in multimodal reading by integrating the various modes at interplay in the texts. For a complete literacy competence, these findings raise the need for engaging learners in deciphering texts from a multimodal perspective by establishing connections across the semiotic resources present in a text.

This study argues that voice is a communicative tool used in people’s daily life, and it can also be a vocal apparatus for actors to deliver acting functions in films. In this study, the author demonstrated how actors project voice qualities of pitch, loudness and intonation to achieve various communicative functions in vocal acting by investigating four selected Hollywood films. The speech analysis tool Praat was used to process actors’ voice qualities in the selected data to verify the argument. After investigation, this research discovered that actors can use voice qualities to represent information and emotion and indicate characters’ attitudes, relationships and situations. Moreover, this research revealed that actors’ voice is more dynamic than other acting participants (such as makeup, costume).

Rap battles are a growing phenomenon in China, but few studies have examined their interactional structure, especially from a multimodal perspective. In this paper, I conducted a microethnographic analysis of the sequential and simultaneous co-operation in Chinese rap battles from the national finals of Iron Mic . Results showed that in sequential co-operation, rappers accumulated the pre-existing end rhymes and created situated rhyming patterns in different rounds of battle performances, which helped them to outflow their opponents. Rappers also transformatively adapted lyrics and bodily movements from their opponents, demonstrating their ability to improvise and their superior lyricism over others. Moreover, battle emcees simultaneously used hand gestures to rebut their opponents’ accusations or show respect to the rap community. The study demonstrates that co-operative action is a viable theory to account for the interactional structure in rap battles and potentially other similar genres, highlighting the importance of a multimodal approach to social interactions.