A Multimodal Study of Doctors’ Responses in Free Online Medical Consultations

4.1.1 Head Movements and Vocal Particles during the Repetition of Questions

The former questions are constructed by several disjointed phrases, nouns, or mere digits, indicating the patients may have undergone certain physical examinations or have a relatively comprehensive understanding of their own medical history; whereas the latter ones usually consist of one or two short interrogative sentences which depicting vague feelings or questioning about a specific symptom instead of the detailed conditions of the patient.

In FOMCs, doctors typically utilize head movements (henceforth HMs), vocal particles (henceforth VPs) as responses to display their uptake to the first type of questions during the reading of the them. The HMs such as nodding and VPs such as “hmm” are defined as “general responses” which are not tied to the meaning of any particular narrative or point in a narrative (Bavelas and Gerwing 2011). In our corpus, however, these two responses often occur after the turn-construction units (henceforth TCUs) of the questions, and sometimes they are synchronized, such as in Extract 2:

Extract 2 (Corpus ID: OP_L_X22)

Figure 3 HMs and VPs at TRP

Translation:

Assistant: Then, the patient says, well, “Now, I have myopia of 800°, and the physical examination showed that I have macular cleavage. Then I just want to ask if surgery is needed for this condition.”

Doctor: Ask him how old he is.

In the question above, the patient first provided two pieces of information: his myopia degree and macular cleavage and proposed his question at the end of the sentence: Should he undergo surgery based on the above conditions. However, in the patient’s original text, he did not clarify who was the subject of the surgery. During the assistant’s repetition of the question, the doctor made HMs and VPs responses between each turn. It was only when the final interrogative sentence was delivered that the doctor opened his lips to prepare for the response.

A turn is usually composed of one or more TCUs (Sacks et al. 1974) and accordingly, each TCU is a coherent and complete expression, and the establishment of a TCU presupposes the completion of the turn and further projects the establishment of a transition relevance place (henceforth TRP) (Clayman 2013). Obviously, the patient’s text-based question can be regarded as a complete turn, but the assistant’s repetition split it into three turns resulting in the doctor made multimodal responses at every TRP. Indeed, in FOMCs, the multimodal responses used by the doctor to display uptake stem from the pragmatic need to interact with the assistant, and this pragmatic need is closely related to the semantic and syntactic structure of the question raised by the patient. The higher-level action of consultation with patients and that of the co-live broadcasting with assistant are influenced by each other.

In Extract 2, the patient had obtained the physical examination results before this consultation. Therefore, the patient was in a k+ status in the epistemic gradient of their own medical condition and history, and further demonstrated a k+ epistemic stance through the relatively detailed question. The doctor responded to each piece of information provided by the patient through nodding and “hmmm” which, according to Deppermann (2013), expressing confirmation. Consequently, the epistemic gradient between the two parties was gradually balanced. However, it can be seen from line 4 that the information provided by the patient was still insufficient. Instead of following the rule of “question-answer” adjacency pairs, the doctor asked a follow-up question about the patient’s age. As shown in Extract 2, before the assistant had fully repeated the question, the doctor had already opened his lips as a preparation to ask about the patient’s age. It can be seen at the end of line 3 that there was a certain degree of overlapping between the assistant’s repetition and the doctor’s vocal preparation. Being at the k+ status in the epistemic gradient of ophthalmic health, the doctor judged that without the patient’s age information, it would be difficult to advise the patient on whether to undergo surgery or not. Therefore, the doctor instructed the assistant to request the age information from the patient. The doctor established their identity as an expert (Ma and Gao 2018) and further held the floor by simultaneously demonstrating a k- stance in the patient’s personal information and a k+ stance in the field of ophthalmic knowledge.

4.1.2 Body Postures After the Repetition of Questions

Doctors also utilize body postures (henceforth BPs) and gaze (henceforth GZs) to display uptake, especially when they receive relative short questions. The constant transition of GZs is clearly shown in Extract 1 while BPS usually preceded or accompanied doctors’ verbal responses, occupying the turn-initial position. Most of the doctors in FOMCs sit behind a desk on which their arms are placed, while cameras and assistants are usually in front of them. If the doctor lean forward when listening to question, they may lean backward as a signal of their turn initiation, as shown in Extract 3:

Extract 3 (Corpus ID: GD_L_x5)

Figure 4 The change of BP

Translation:

Assistant: What are the treatment methods for ovarian adhesions?

Doctor: For ovarian adhesions, first, we need to clarify whether treatment is necessary.

As demonstrated in Extract 3, in line 1 when the assistant was reading aloud the question the doctor maintained a forward-leaning posture, with her hands placed on the table before her. After line 1 a silent interval of roughly 1 s occurred during which the direction of the doctor’s gaze cannot be identified in line 2. In line 3, the doctor inhaled gently, and her gaze refocused on the camera, indicating the completion of her thinking. Roughly 0.5 s later, her posture shifted to a reclined position and she initiated her responsive turn simultaneously.

From the perspective of the epistemic gradient, the essence of the doctor’s posture changes indicates the transition from the K- to K+ status: the question posted by the patient was too short and vague to be responded instantly so the doctor needed 1.5 s to organize her thoughts. Therefore, during the silence, she remained in the forward-leaning posture (LF) which corresponded to the cognitive processing phase, marking the transition from K- to K+ status. Finally, the backward-leaning posture (LB) signals the doctor’s entry into the K+ status as her forearms left the desk and rested naturally. With her upper body fully relaxing on camera, the doctor thereby establishing an “authoritative posture” for the subsequent professional response. This process also coordinates the two higher-level actions of co-live broadcasting and consultation simultaneously: the forward lean is a response to the assistant’s background action of conveying the question, while the backward lean shifts to the foreground consultation action with the patient. The posture freezing during the transition phase avoids the abruptness of action shifting and maintains the fluency of interaction in the digital context. Notably, such significant BPs changes (from forward to backward leaning) mostly occur when patients pose short questions with vague information; in contrast, for the detailed long questions such as in Extract 2, doctors mostly maintain a neutral sitting posture and only display uptake through subtle HMs and VPs. This indicates that the visual intensity of body postures is negatively correlated with the information sufficiency of patients’ questions: the vaguer the information, the more significant the posture changes. This helps compensate for the delay in verbal uptake and ensures the effective balance of the doctor-patient epistemic gradient.

4.2.1 Beat Gestures for Listing

Since doctors in FOMCs sit behind desks, their arms and heads are typical gesture resources that they can use. Sometimes both modes are used to list all the possible diagnosis and treatment in the form of beat gestures, such as in Extract 4:

Extract 4 (Corpus ID: GY_H_X10)

Figure 5 GEs and HMs in the line 2 and line 3

Figure 6 GEs and HMs in the line 4 and line 5

Figure 7 GEs and HMs in the line 6 and line 7

Figure 8 GEs and HMs in the line 8 and line 9

Translation:

A: Well, for a 37-year-old woman with an AMH level of 0.75 ng/mL, 2–3 follicles on each ovary, and normal sperm quality in the male partner, what ovarian stimulation protocol do you think is suitable?

D: It doesn’t just depend on my assessment alone; this also needs to be carried out based on her own monthly condition. Of course, you can use the antagonist protocol if you want, and it’s also acceptable to use the PPOS (Progestin-Primed Ovarian Stimulation) protocol to accumulate embryos. However, it still needs to be determined by combining her specific condition that month, her hormone levels, her height, weight, and other factors such as her comprehensive medical history.

In Extract 4, the patient first provided some physical examination information and asked the doctor which ovulation induction protocol was suitable for her. In line 2, the doctor did not respond to the projection of treatment advice directly. Instead, she provided a dis-preferred answer, in which is structurally different from preferred answer, as the dis-preferred is designed to incorporate with inter-turn and turn-initial delays, prefaces, accounts, and mitigations (Lee 2013), that her own assessment could not serve as an appropriate recommendation. In this case the dis-preferred response is followed by the latter two. In line 3, the doctor made her first account that her diagnosis needed to be based on the patient’s specific monthly situation. It is the exact time when she started to utilize gestures (PRE1) as her hands tapped each other with vertical palms (PVT1 and PVT2). They did not return to their home position but were frozen in the post-stroke hold phase (PH1) at the end of the turn, indicating that the doctor still had her speakership (Sacks and Schegloff 2002). In line 4 and line 5, she still listed two relatively common ovulation induction protocols: the antagonist protocol and the PPOS protocol (Deng et al. 2025). According to multimodal transcription, the utterances about these two protocols were accompanied by two mutual taps with palms downward (PDT2 and PDT3) as well as two nods (N7 and N8). These two lines served as a mitigation to the dis-preferred response and also laid the groundwork for the next four lines as accounts. In line 6, 7, 8, and 9, the doctor restated her view that a comprehensive diagnosis needed to be made based on the patient’s personal specific conditions and further listed these conditions: her hormone levels, height, weight, and “other factors”. While saying these four noun phrases, the doctor respectively performed mutual a hands-tap with palms upward (PUT), an outward turning of both hands (HTO2), a hands-holding gesture with palms upward (PUH), and a hands-waving with palms upward (PUW2), and each gesture was accompanied by a nod (N15, N16, N17, N18). In the final summary stage, when the doctor said, “a comprehensive medical history”, her gestures entered the recovery phase (REC).

4.2.2 Deictic Gestures for Embodiment

As a carrier of illness, the patient’s body has always been the foundation of topics in doctor-patient consultations. While some patients are often in a k+ status regarding their own medical history, they are in k- status with general human anatomical. To provide enough medical information, doctors not only use beat gestures to list diagnosis or treatment, but also employ deictic gestures to explain recondite concepts or ideas in their verbal responses. Specifically, we found that doctors mainly rely on anatomical models or their own body parts to illustrate the site of condition, as shown in Extract 5 and Extract 6.

Extract 5 (Corpus ID: OR_B_X5.1)

Figure 9 GEs with a model in line 1, line 2, and line 3

Figure 10 GEs with a model in line 4, line 5, and line 6

Translation:

Many fractures are often diagnosed by doctors through palpation and experience: doctors feel the location of the fracture through the skin, muscles, and ligaments, and then use some functional changes to further (a word hard to identify) assess the reduction status of the fracture. This is often a functional reduction.

Extract 5 is an excerpt from an orthopedic doctor’s consultation. The consultation started from a patient’s question that whether a surgery was needed as he had a left radial head fracture with anterior dislocation of the elbow joint, along with nearby bone fragments. During his explanation, the doctor introduced two treatment approaches: anatomical reduction from Western medicine and functional reduction from traditional Chinese medicine. To illustrate how traditional Chinese doctors conducted functional reduction examinations without technological tools such as CT scans in the past, he used a medical anatomical model near him. Interestingly, the model he showed was not of the elbow joint, but of the spine. In the excerpt from Extract 5, the doctor did not mention specific body parts at all from the moment he fetched the model (MF) to when he (MR); the model’s purpose was to demonstrate the palpation technique used in traditional Chinese medicine. In line 2, the doctor introduced that to locate patients’ fractures, traditional Chinese doctors used hands to palpate skin, muscles, and ligaments respectively. He noted that as attached directly to bones ligaments were much closer to bones than the other two. Therefore, when talking about the “skin” and “muscles”, he used the model as a baton to preform two beat gestures (MW1 and MW2), and his left hand’s fingers were tightly pinching it as in Image 3. In contrast, when referring to “ligaments,” he demarcated its range on the model (MD). Thus, this gesture that pointing the physical location of the tissue can be classified as a deictic one (Norris 2004). Moreover, as shown in Image 4, the doctor specified the meaning of his verbal response as his left hand maintained a short distance from the model, which precisely simulated the inherent thickness of the ligaments themselves (Kendon 2004). In line 3, to emphasize the palpation method of “feeling,” the doctor once again pinched the model with his left hand’s fingers as in Image 5 (MP2).

Since the patient in Extract 5 did not provide age information, the doctor responded him with the information maximization strategy. He not only presented two treatments from the perspective of traditional Chinese medicine and Western medicine, but also pointed out that whether to choose surgery needed to take the patient’s age into account. He further contended that young people had greater demands for elbow joint function, making surgery a better option. For elderly patients, if the basic function of the elbow joint was preserved, it was not recommended to take the risk of surgery. When defining the basic functions of the elbow joint, the doctor used his own right arm as a multimodal resource to give an embodied response, as shown in Extract 6:

Extract 6 (Corpus ID: OR_B_X5.2)

Figure 11 GEs by the doctor’s right arm

Translation:

D: A radial head fracture mainly affects two aspects: one is the flexion and extension of the elbow joint, and the other is the rotation function of the forearm.

As can be seen from the transcription, in line 1, the doctor’s gestures entered the preparation phase only after he finished saying “radial head fracture”. However, before mentioning the two basic functions of the elbow joint in his verbal response, the doctor had already completed a rotation movement (FR1). In line 2, when introducing the first basic function, the doctor’s elbow joint flexion movement (EF) occurred prior to the corresponding speech content. In line 3 and line 4, the doctor performed three more sets of forearm rotation movements (FR2, FR3, FR4) before the phrase “rotation” was mentioned. Based on this, we can conclude that when doctors use embodied multimodal resources to respond, these resources have their own structure and often appear prior to verbal expressions.

Moreover, embodied multimodal resources may differ from speech not only in terms of sequence but also in terms of pragmatic meaning. In the doctor’s verbal response, it was pointed out that radial head fracture can affect two basic functions of the elbow joint: the flexion and extension function of the elbow, and the rotation function of the forearm. Additionally, the doctor made an embodied response by using his own right arm. Unlike the verbal content of the response, what the doctor demonstrated was not the basic functions of the elbow joint affected by radial head fracture, but those of a healthy person. As shown in Image 6 and Image 7 respectively, the doctor’s elbow joint possesses these two basic functions. In Extract 6, the doctor used his own right arm to demonstrate the speed, force, and maximum range of angles of the movement a normal elbow joint could present. Again, by utilizing nonverbal multimodal resource, the doctor conveyed more information that was not mentioned in his verbal response.