Instructing embodied knowledge: multimodal approaches to interactive practices for knowledge constitution

1 The transfer of practical knowledge

The acquisition of knowledge, both conceptual and practical, is the cornerstone of all types of instruction, and in fact of societal dynamics as a whole.^[1] Apart from the study of the product of acquisition, i.e. demonstratable knowledge, uncovering the dynamic processes through which this knowledge is passed on can provide valuable insights into (collaborative) instructional practices as well. In fact, recent work in conversation analysis, interactional linguistics and sociology has focused on the acquisition and transfer of practical knowledge (e.g. knowledge related to the body, to movement and to ‘doing things’ understood as skills), zooming in on the intrinsically social nature of these activities. Based on the empirical analysis of various instructional settings, it has been demonstrated that participants interact and collaboratively organize the transmission, acquisition and constitution of knowledge using different verbal and nonverbal resources. Among the phenomena that have been studied are music (Haviland 2007; Reed 2015, 2020; Reed and Szczepek Reed 2013, 2014; Sambre and Feyaerts 2017; Stevanovic 2017; Szczepek Reed et al. 2013), dance (Broth and Keevallik 2014; Evola and Skubisz 2019; Keevallik 2010, 2014b, 2015; Kolter et al. 2012; Müller and Bohle 2007; Müller and Ladewig 2013), sports (Evans and Fitzgerald 2017; Evans and Reynolds 2016; Keevallik 2020; Okada 2013; Singh 2019; von Wedelstaedt and Singh 2017), self-defense and martial arts (Råman 2019; Råman and Haddington 2018; Schindler 2011, 2016; Stukenbrock 2014a, 2017), driving and flying (Deppermann 2018a, 2018c; De Stefani and Gazin 2014; Helmer 2021; Levin et al. 2017; Melander and Sahlström 2009; Mondada 2018), cooking (Mondada 2014a; Raevaara 2017), handicraft (Heinemann and Möller 2015; Lindwall and Ekström 2012), dentistry (Hindmarsh et al. 2011, 2014; Lindwall and Lymer 2014; Rystedt et al. 2013), surgery (Bezemer et al. 2011; Mondada 2014b, 2014c, 2014d, 2014e; Zemel and Koschmann 2014), vocational training (Filliettaz 2007; Filliettaz et al. 2010) and visits in museums (Kesselheim 2012).

When dealing with practical knowledge, a key feature to be taken into account is that much of this knowledge is bound to the body, and may therefore be implicit (Fuchs 2012; Ryle 1949) rather than of an explicit conceptual nature that can be easily verbalized (see Section 2 below). As a consequence, instructors frequently rely on elaborate multimodal practices, for instance, bodily demonstrations or simulations of the activity at issue (Clark and Gerrig 1990; Goffman 1986 [1974]; Müller 2014; Putzier 2012; Streeck 2009; cf. also Clark 2016b). Demonstrations are usually not simple ‘nonverbal’ performances or displays of the knowledge to be transferred, but highly structured social activities of sharing and distributing conceptual knowledge adjusted to their instructional purpose. Apart from demonstrations, instructors obviously also rely on other practices, like corrections and different kinds of directives. It is common to those practices that bodily and verbal resources are tightly coordinated to build multimodal gestalts (Mondada 2019), e.g. holistic patterns with a temporal structure involving different semiotic resources that are possibly coordinated across participants. In addition, since instructions of embodied knowledge are not only bound to the body and space but also to time, systematic patterns can be observed on different scales, ranging from the micro-timing of different modalities on the level of single utterances up to large sequential and interactional patterns.

Next to a social-interactional approach to practical, embodied knowledge, other views on embodiment and/in language have been proposed, which may generate valuable insights into the processes of instruction as well. A major strand in cognitive science, for instance, deals with the foundations and ramifications of embodied cognition (Clark 1997, 2016a; Lakoff and Johnson 1999; Pecher and Zwaan 2005). This view holds that many aspects of (human) cognition are embodied in the sense that they are rooted in and dependent on the cognizer’s physical body. In linguistics, this has been demonstrated in the large body of research on metaphor and image schemas, which has its roots in early work by George Lakoff and Mark Johnson (Johnson 1987, 2018; Lakoff and Johnson 1980). What the significant body of research on this topic has shown is that much of abstract thinking and abstract linguistic knowledge depends on corporeal experience. In fact, it remains an open question whether abstract conceptual knowledge can be fully separated from practical knowledge, given the strong interrelatedness between the two. In addition, instructional activities often aim at making the implicit more explicit and at transforming the practical into conceptual knowledge. Furthermore, one can assume further layers of knowledge that are not directly accessible, e.g. knowledge about, for instance how a certain type of dance usually looks, its history, the positions and roles participants take on etc. In the case of the embodied practical knowledge that we focus on here, however, the relationship between the body and ‘knowing’ is more direct, and in this sense, it may be interesting to explore the relationship between preconceptual experience, embodied cognition and embodied knowledge.

This thematic issue on Instructing embodied knowledge presents a collection of concise empirical studies – very much in the spirit of the Linguistics Vanguard journal – that illustrate the ongoing work on the construction, transfer and communication of practical knowledge. Although these contributions are informed by different theoretical and methodological frameworks, including the ones mentioned above, they are united in a common focus on the intrinsic multimodal nature of these activities, combining linguistic descriptions and bodily-spatial analyses. Taken together, these studies provide an overview of recurrent questions, challenges, commonalities and differences in the different approaches to embodied instruction. This introduction serves to situate these approaches and highlight the particular focus of the individual contributions. In the next section, we discuss some of the challenges related to (the instruction of) embodied knowledge, on the basis of which we formulate a set of key questions that guide the contributions to this special issue, but arguably also much of the research on this topic as a whole (Section 3). In Section 4, we briefly describe how the individual contributions relate to these key questions and formulate the main results of these studies.

2 Embodied knowledge in (inter)action

As we mentioned above, it is a widely held position that a large body of knowledge is tacit. This idea is captured in Polanyi’s well-known quote “We know more than we can tell” (Polanyi 1966). By distinguishing between knowing and telling, a binary opposition is created that is reflected in numerous works, including studies on embodied knowledge:

1. Tacit knowing vs. explicit knowledge (Polanyi 1966).

2. Knowledge by acquaintance vs. knowledge by description (Russell 1910–1911)

3. Knowing that vs. knowing how (Ryle 1949)

4. Implicit vs. explicit memory (Schacter 1987)

5. Practical/performative vs. semantic knowledge (Foppa 2002; Wulf et al. 2001)

6. Procedural vs. declarative knowledge (Anderson 2007; Engelkamp and Zimmer 2006)

When applying such oppositions to the type of ‘embodied knowledge’ that is central to the present thematic issue, it is apparent that we are dealing with knowledge that needs to be enacted by the lived body in order to be accessible in the interaction (be it in dance, instrument playing, singing, sports or any other type) and in this sense defies its separation from that same body as its original habitat. In other words, it is a form of knowledge that is non-representational in the sense that it can only be partly ‘represented’ at the conceptual level (cf. declarative/semantic knowledge) without resorting to the lived body: ‘the knower’ and ‘the known’ are inseparably intertwined. This basic characteristic of embodied knowledge can be related, at least in part, to its holistic character, meaning that different modalities of perception and experience are integrated with each other to build ‘gestalts’. In other words, we are dealing with “forms of knowing that are based on intermodal and sensorimotor gestalt units, that means, they integrate different sense modalities and bodily movements into a holistic experience” (Fuchs 2016: 219–220). Important to note, these gestalts usually exhibit a particular temporal or procedural structure, in which the different interacting resources may be guided or constrained by different temporalities (Deppermann and Günthner 2015; Deppermann and Streeck 2018; Mondada 2019). That is to say, certain (non-)verbal actions tend to occur simultaneously whereas other actions occur prior to others, thus projecting and somehow constraining the appearance of a following action in the gestalt. In other words, the integration of different modalities as part of a holistic structure does not presuppose strict simultaneity. In addition, embodied knowledge does not only pertain to individual actions performed by individual bodies, but in many cases is also inextricably bound to bodily actions by others. The notion of intercorporeality, which was coined by Merleau-Ponty (1945, 1964, has recently received increasing attention in research on social cognition and embodied interaction (Meyer et al. 2017; Tanaka 2015). It highlights the grounding of intersubjectivity in the bodily presence of self and other(s), thus providing a framework for the analysis of the bodily nature of social interaction.

Of course, such binary distinctions between types of knowledge – e.g. knowing that and knowing how – should be interpreted as cover terms for highly diverse fields of knowledge. Regarding body knowledge, several suggestions have been made to systematize different forms of knowledge (cf. Casey 2000; Fuchs 2017). Fuchs (2012) distinguishes six forms of memory of the body: procedural, situational, intercorporeal, incorporative pain and traumatic memory. Procedural memory arguably is the kind of body knowledge that is most often referred to, since it comprises “patterned sequences of movement, well-practiced habits, skillful handling of instruments, as well as familiarity with patterns of perception” (Fuchs 2012). It is also this kind of knowledge that is most subject to explicit instructional activities. It is important to note that Fuchs does not only include the bodily actions into the notion of body knowledge but also different kinds and patterns of perception. From an interactional perspective, the importance of perception in practices has been highlighted by Charles Goodwin (1994, 2001, 2003 in his work on professional vision (Nishizaka 2017; cf. Melander 2018). But of course, besides seeing, other kinds of perception like smelling, tasting and feeling may form an integral part of social practices and the respective body of knowledge (Jenkings 2017; Mondada 2019, 2020; Nishizaka 2011, 2014). In instructing body knowledge, the perceptive dimension deserves particular attention, since the instructors face the task to instruct these aspects as well.

The largely procedural and intercorporeal nature of embodied knowledge is a particular challenge for linguists aiming at a systematic analysis of embodied interaction. How can linguists deal with information that we can generally not express by speaking? In fact, when looking at the literature on the transmission of embodied knowledge, we find notions such as “stumme Weitergabe” ‘tacit transmission’ (Schmidt 2008) and ‘unvoluntary exhibition during performance’ (Schindler 2011) to characterize the process. Some authors even characterize certain kinds of performative knowledge as something “das nicht sprachlich übermittelt, sondern nur am eigenen Leibe erfahren werden kann” (‘that cannot be transmitted through language, but rather only experienced through one’s own body’) (Fischer-Lichte 1999: 10). One particular locus of investigation that may be of interest to linguists is the acquisition and transfer of embodied knowledge, for instance in teaching, training and/or coaching situations, because this setting often requires forms of (verbal) explicitation or demonstration of the knowledge to be transmitted and acquired. What emerges is an interesting tradeoff (or tension) between two major ways of acquiring embodied knowledge: implicit “learning by doing” and experiencing on the one hand and explicit “learning by synthesizing” on the other. An interactional-linguistic account may provide valuable insights into the relationship between these learning/teaching processes by approaching the construction and instruction of embodied knowledge as (i) a social activity, (ii) involving the deployment of different semiotic resources, and (iii) using different techniques and devices for the communication of knowledge.

Studying the instruction of embodied knowledge as a social activity implies a focus on the collaborative organization of the ‘learning activity’, as has been shown among others in the context of driving instruction, dance instruction, dentistry training, surgery training, self-defense training and martial arts training sessions, among many others (for references see introduction above). It is argued that instruction in those contexts is organized by communicative practices. Following Deppermann et al. (2016) the notion of (communicative) practice highlights the materiality of the communicative situation and the use of multimodal resources in gestalt like patterns with a temporal structure. Furthermore, practices are typically related to certain participation frameworks (e.g. instructor(s) and learner(s)) and social actions (e.g. correction, request, assessment). Practices are routinized patterns that are reflexively bound to contexts, due to their indexical nature. In addition, practices are typically bound to certain historical contexts. It has also been shown that practices may not only be domain specific but also tend to occur across settings to which they are then adapted (a.o. Stukenbrock 2014b).

Second, adopting a multimodal perspective on this activity requires all different semiotic resources to be treated on a par, including gesture, posture and eye gaze (cf. Arnold 2012; Filliettaz 2007; Nishizaka 2007, 2011, 2018; San Diego et al. 2009). Taking into account the above-mentioned tradeoff/tension between verbal explicitation and implicit learning by doing, a multimodal approach is particularly interested in the tight temporal coordination of bodily and verbal resources in instructional sequences. This coordination can be studied at the micro-level of single utterances as well larger sequential-interactional patterns (Broth and Keevallik 2014; Keevallik 2013, 2015; Levin et al. 2017; Mondada 2014b; Reed and Szczepek Reed 2013; Szczepek Reed et al. 2013).

And thirdly, apart from the social and multimodal nature of the activities at hand, research on the instruction of embodied knowledge needs to zoom in on the techniques and devices that are used. These include descriptions (as a form of explicitation), directives (which can be verbal as well as nonverbal) (Deppermann 2018b; De Stefani 2018; Okada 2018; Rauniomaa 2017; Stevanovic 2017), demonstrations (Evans and Lindwall 2020; Goffman 1986 [1974]; Keevallik 2013, 2014b; Mondada 2011; Putzier 2012, 2016; Råman 2019; Rystedt et al. 2013), related concepts such as depictions, simulations and iconic representations (Clark 2016b; Hindmarsh et al. 2014), the use of embodied metaphors (Kolter et al. 2012; Müller 2014; Müller and Ladewig 2013), the creation of multimodal viewpoints (Sambre and Feyaerts 2017), correction techniques and assessments (Evans 2017; Evans and Reynolds 2016; Keevallik 2010; Levin et al. 2017; Muntanyola-Saura 2015; Weeks 1996), the use of voice (Keevallik 2020), and on the learner’s side for instance practices of displaying understanding (Hindmarsh et al. 2011), the involvement in joint imagination (Keevallik 2014a; Nishizaka 2003; Stukenbrock 2014a, 2017; von Wedelstaedt and Singh 2017), etc.

3 Key questions

Based on the brief orientation sketched above, which has shown that the instruction of embodied knowledge may be of interest to several (sub)disciplines, we can identify a set of key questions that may help to shape the discussion. The contributions to this special issue address (and hopefully help to answer) some of these questions, but we hope that they may also serve as a ‘mission statement’ for research on this topic, shaping the agenda for future studies as well. In general, the questions can be clustered in three groups. A first set of questions deals with the way in which instructions are construed, both in terms of the semiotic resources involved and the temporal-sequential organization. A second set focuses on demonstrations as a central practice and the potentially interesting relation between demonstration, rehearsal and actual performance. A third and final set reflects on the relationship between perceptual access, sensation and the acquisition of embodied knowledge.

Multimodality, Temporality and Context in Instructions

1. How are demonstrations and other kinds of instructions organized in time at different levels of granularity/within units of different sizes (utterance, sequence, phase)?

2. What is the relationship between the verbal and the bodily level in instructional interaction? For example, which verbal, vocal and nonverbal resources are used and which are their functional profiles? How are descriptive, iconic and deictic resources combined and alternated? For example, can a nonverbal depiction ‘take over from’ a purely verbal description? Which aspects of meaning does the verbal level ‘add’ to a bodily demonstration?

3. Can we find ‘multimodal gestalts’ in instructions, e.g. relatively stable patterns that are constant over different contexts? How are such patterns adapted to, for example, different skills/activities, participation formats and group sizes?

Demonstration, Rehearsal, and Actual Performance in Instructional Settings

1. How can we capture the continuum between the demonstration/simulation of an activity and its actual performance/doing? E.g. the difference between demonstrating a dance step, practicing this step and actually dancing.

2. To what extent are such differences interactionally relevant for the participants involved? Do participants signal (gradual) differences between demonstrations and actual doings of an activity in interaction?

3. Can demonstrations be realized collaboratively, e.g. as multimodal co-constructions? And if so, what are the roles of the two demonstrators?

Seeing, Experiencing, and Knowing in Instructions

1. What is the relation between seeing and doing in instructional settings? As part of the instructional interaction, students/learners often first see the demonstration at hand. How is the visually accessible information structured verbally?

2. How do instructors distinguish between visible or otherwise perceptible skills, and communicate sensations and experiences that are not visibly accessible, like body internal sensations, intercorporeal sensations, haptic sensations, sound and music qualities and tastes? How is the acquisition of professional perception accomplished?

3. How is an increase of knowledge reflected in such activities? How can we provide interactional evidence of the learning effect of such practices? How do instructors adapt their interventions due to (non-)progressions in ‘knowing’? Which are the implications that can be drawn for improving the transmission of embodied knowledge?

Important to note is that cutting across these sets of questions, there is an ongoing discussion on terminological and conceptual distinctions relating to embodied knowledge. The papers in this special issue contribute to this discussion and, reflecting the different perspectives we have collected on the phenomenon, represent – sometimes radically – different positions. For instance, in the questions above, we have resorted to concepts such as demonstrations, depictions, simulations and iconic representations, which roughly refer to the same (set of) phenomena but may differ in scope or definition, depending on the particular theoretical framework in which they are used. Rather than trying to resolve the terminological confusion in this introduction, which would warrant a contribution of its own, or take a stance in the discussion, which would go against the broad spectrum we aim to present, we refer the reader to the individual papers for further discussion. We present the key ideas of these contributions in the following section.

4 Contributions to the special issue

Setting the scene, the opening paper of the issue distinguishes between different kinds of learnables in instructing embodied knowledge (Szczepek Reed). A first set of papers deals with different practices of demonstration or depiction (Hsu, Brône and Feyaerts; Keevallik; Ehmer; Singh). The second set of papers focuses on more conceptual practices, making use of concepts like image schemas, metaphors and metonymy (Sambre; Stevanovic). The issue concludes with two papers that widen the scope and deal with the role of experience and epistemic access in gaining, instructing and sharing knowledge (Reed; Kesselheim and Brandenberger).

Beatrice Szczepek Reed distinguishes two kinds of ‘learning goals’ or ‘learnables’ in vocal masterclasses. While body-focused learnables concern physical skills, concept-focused learnables concern mental processes and concepts. While the author acknowledges that all learnables in musical classes are both conceptualized and embodied, she shows that the two mentioned learnables are instructed differently. In the instruction of body-focused learnables the body itself is oriented to as the ‘place where the desired action originates’, by using for examples visual access to the student’s body, touch and material objects as interactional resources. Concept-focused instructions are instructed though the body as well but the body plays a similar role as in much of human interaction, and it is used as a vehicle for communicating ‘invisible processes’. Resources employed here are for example depictions of ‘to-be-imagined’ mental states and explicit lexical references to concepts.

Hui-Chieh Hsu, Geert Brône & Kurt Feyaerts use the concept of depiction, as it was recently spelled out in Clark’s Staging Theory (2016), to zoom in on instructional sequences in which meaning is communicated by resorting to nonverbal rather than verbal resources. More specifically, they single out the phenomenon of speech-embedded nonverbal depictions (Hsu et al. 2021), which revolves around the staging of physical scenes, using gestures, facial expressions, vocalizations, gaze, etc. for others to imagine the scene depicted. Based on instances collected in cello master classes, they discuss patterns of multimodal iteration, where roughly the same meaning is communicated multiple times (possibly for instructional purposes), using different semiotic resources and/or signaling methods. It is shown how this phenomenon can be treated using insights from dialogic syntax (Du Bois 2014), a model developed for the analysis of parallelisms and affinities across turns in conversation.

Leelo Keevallik analyzes the role of vocalizations in the instruction of bodily skills in the particular context of jazz dance teaching. Using sequences of multiperson interaction, involving a teacher working with a large group of students, she shows that vocal resources other than entrenched (lexical) symbols can be used to express simultaneous body movement. Vocalizations present an interesting case for the emergent nature of meaning, arising in and as part of the interaction, and conveying indexical, embodied and sensory rather than conventionalized information. At a methodological level, she argues that Conversation Analysis provides a suitable framework for the analysis of the transfer of embodied knowledge in (inter)action.

Oliver Ehmer focuses on demonstrations as a resource for instructing embodied knowledge. The author highlights the fact that demonstrations are social activities in which not only the instructor but also the learners may participate. He focuses on one way of co-participation, namely students synchronizing with the actions of instructor. Based on a corpus of dance classes in Argentine Tango two distinct practices of synchronization in demonstrations are analyzed. In ‘orchestrated synchronizations’ the instructors actively pursue the student’s synchronization, which typically happens for the instruction of new knowledge. In ‘emergent synchronizations’, in contrast, the instructor rather invites than requests the students’ synchronization, a practice which is typically used to realize corrections. The two practices of synchronization, the author argues, should be considered as prototypes with possible transitions along a continuum, constituted by various criteria such as timing, bodily-spatial formation, and direction of synchronization.

Ajit Singh approaches the question how embodied knowledge is socially mediated and communicatively constructed by studying embodied action plans in training sessions in trampolining. Action plans can be understood as the embodied equivalent of projections as described in Conversation Analysis for talk-in-interaction, referring to complex embodied practices through which actors prepare and coordinate further actions, in a way that is observable and thus rooted in a social and communicative project. The sequential analysis of a longer exercise unit in trampoline training shows how these embodied action plans are used by both athletes and coaches for instructing, by making use of pre-enactments (i.e. demonstrations), relevant for the subsequent performance of an exercise. The social dimension of such pre-enactments becomes apparent as well because of their role in the production of intersubjectivity: embodied action plans are directed towards a spatially present and perceptible ‘other’ engaged in the process of knowledge communication.

Paul Sambre in his contribution challenges the concept of image schemas, a term used in cognitive linguistics to refer to embodied prelinguistic structures that motivate patterns in language and thought, when confronted with the instruction of embodied practices. More specifically, he argues that the notion of high musical pitch, conceptualized in terms of an abstract embodied image schema, needs to be replaced by a multidimensional body schema, taking into account the interplay between speech, gesture, the performing body and the musical instrument (in this analysis the trumpet). Rather than adopting the classical view on high pitch in terms of upward verticality, he argues that this conceptualization needs to be enriched by other dimensions such as nonvertical movement or immobility, which better reflect the natural corporeal reality of the embodied knowledge to be acquired. Such enriched schemas may provide a more dynamic and flexible resource on which both teachers and students can rely in the process of knowledge transmission.

Melisa Stevanovic focuses on the use of noun metaphors of bodily behavior in the instruction of children’s music instrument playing. Combining insights from Conversation Analysis and Conceptual Metaphor Theory, she presents a sequential analysis showing the temporal relation of the metaphorical elements in the teacher’s instruction and the children’s handling of the instrument. A close reading of video-recorded sequences shows that metaphors may serve as initial orientation points, through which teachers can assess students’ knowledge and through which students can demonstrate that knowledge. At the same time, the analysis reveals that metaphor turns may help teachers in initiating correction in a complex movement sequence, but also in the formulation of an affirmative evaluation of students’ performance. In this sense, metaphors may serve as an important instructional resource in the transmission of embodied knowledge, transforming experiential features into an intelligible behavioral gestalt.

Darren J. Reed analyzes how instructors manage to claim that they have knowledge about the internal workings of a student’s body. The internal workings of one’s own body and ‘how the body feels’ are a domain of experience that inherently belong to the epistemic territory of that particular person. It is thus a practical problem for instructors to claim knowledge about this domain, in some cases even to know better than the person herself. Based on data from musical master classes, in which a student is instructed in front of an audience, the author analyzes how an instructor of Alexander technique deals with this problem. This is realized through a sequential development from an intimate contact to the student, involving interpersonal touch, to a public display for the audience, using mimesis. In his contribution the author makes a point in how Conversation Analysis may be used to methodologically deal with ‘invisible sensorial interactions’ between participants and how those interactions are integrated into multimodal patterns of sense-making.

Wolfgang Kesselheim & Christina Brandenberger zoom in on the experience and transfer of knowledge in the context of science and technology centers. Such centers typically offer hands-on exhibits through which visitors can discover and experience particular natural phenomena. And since most visitors do not visit such centers alone, an interesting question is how individual, often multisensorial experiences are shared with co-visitors, leading to a form of transfer of bodily experienced knowledge in situ. The authors distinguish between two patterns of joint discoveries in this specific setting. In the first type, the co-visitors establish a joint focus of visual attention in their common perceptual space, which in this particular study is measured using mobile eye-tracking data of recorded participants. In the second type, a co-visitor repeats the actions of another visitor that led to a particular discovery, hence creating a form of reproduction sequences that is strongly based on shared experience and intensified intercorporeality.