Depicting handshapes for animate referents in Swedish Sign Language

2.1 Depicting constructions and handshape choices

Ever since the earliest work on depicting constructions in American Sign Language (ASL), which introduced the term classifier for such constructions (Frishberg 1975, Kegl and Wilbur 1976, Supalla 1978), there has been an ongoing debate on how to analyse these signs and which terminology is best suited for them (for an overview, see Lepic and Occhino 2018, Schembri 2003, Zwitserlood 2012). However, the term classifier has been somewhat inconsistently applied as to whether it refers to the whole sign or rather the handshape within it (Liddell 2003, 261–2). In this article, we use the term depicting construction (also used by Cormier et al. 2012, Engberg-Pedersen and Ikeda 2019), based on Liddell’s 2003 term depicting verb, to refer to the entire sign construction. When we refer to the handshape categories within such constructions in the context of this study, we use the labels 1-handshape and 2-handshape, respectively.

Depicting constructions fall on a continuum, ranging from less directly depicting to highly depicting forms. The constructions generally denote an action/motion or location of a referent, and depending on one’s categorisation, potentially also its size and shape. They often incorporate spatial meaning through the use of signing space – that is, the articulation of signs is directed towards or at real or imagined (localised) referents around the signer (Liddell 2003). For example, a depicting construction can mean ‘a person moving past a car’, with one hand representing the person (generally a 1-handshape in STS) and the other hand representing the car (generally a flat hand in STS). However, the same construction could also include a movement or bending of the hand or finger to express the manner of motion, such as ‘walking upright’, ‘walking hunched over’, or ‘running past in an arc path’. If a construction has the meaning of a person or animal standing or sitting (generally a 2-handshape in STS) at a specific location, the location in signing space relative to the signer and any other real or imagined referents can be meaningful, as well as the orientation of the hand when expressing a posture – e.g. ‘sitting up’, ‘lying down’, or ‘hanging upside-down’. Thus, many analyses of depicting constructions have argued that each part of the sign form can be assigned different parts of a compositional meaning. However, it is difficult to differentiate the more lexical signs from more productive depicting signs as belonging to distinct categories. For instance, a lexical sign such as ‘to fall’ (in STS) is listed in the dictionary with a 2-handshape first “standing” and then “falling over” on a flat surface (generally the non-dominant hand is in the form of a flat hand). From this citation form, there are gradient variations of depicting constructions representing a specific referent falling in a certain manner (for different views on this taxonomy, see, e.g. Cormier et al. 2012, Lepic and Occhino 2018, Zwitserlood 2012).

Previous research on depicting constructions has also argued for a distinction between transitive/agentive and intransitive forms, based on whether the selected handshapes represent hands or the handling of instruments (mainly transitive constructions) or rather represent whole entities (mainly intransitive constructions) (Benedicto and Brentari 2004, de Lint 2018, Kimmelman 2016, Kimmelman et al. 2019, Rissman et al. 2020, Simper-Allen 2016, Wallin 1994). This can also be a factor in the viewpoint of the construction, whether the signer is immersed as an imagined participant in the depiction, or rather an outside observer (Dudis 2004, Perniss 2007). Thus, as noted by Benedicto and Brentari (2004) and Kimmelman et al. (2019), the type of depicting handshape is often related to its argument structure. However, Kimmelman et al. (2019) also found that whole-entity handshapes could be used transitively in some cases, and similar findings were also made in studies of STS (Simper-Allen 2013, 2016, Wallin 1994). Only a few studies have analysed the factors influencing the choices of different handshape variants for the same referent in depicting constructions. Zwitserlood (2003) found in data from Dutch Sign Language (NGT; Dutch: Nederlandse Gebarentaal) that there was some variation in the choice of whole-entity handshapes for certain referents, and suggested that this might be due to allophonic or free variation. However, Simper-Allen (2016) showed that in STS, the choice of handshapes when describing an event involving tools (e.g. using a knife for the activity ‘cutting’ or ‘chopping’) might be determined by lexical and grammatical rules, but simultaneously exhibiting different preferences dependent on the signer or the specific tool depicted. For instance, the handshape representing a knife in a depicting construction for an activity like ‘cutting’ or ‘chopping’ can vary between a flat hand (representing the knife blade) or a fist with the thumb on top against the curled index finger (as if holding the handle of a knife) – see also Börstell (2017) for variation and preferences in handling- vs instrument-handshape alternations in STS.

In this study, we focus exclusively on depicting constructions in which the handshape represents the whole entity, specifically an animate referent – human, human-like, or animal. This will be described further in the following section.

2.2 Handshape choices for animate referents

Classifier choices in spoken languages are known to be influenced by animacy (e.g. Adams and Conklin 1973, Allan 1977, Denny 1976, Kiyomi 1992), and the choice of (the type of) handshape (e.g. whether the hands represent the whole or parts of a body/entity) is known to be linked to animacy, agency, and salience across sign languages (e.g. Barbera and Quer 2018, Benedicto and Brentari 2004, Engberg-Pedersen 1993, Kimmelman et al. 2019, Rissman et al. 2020). With regard to depicting constructions with animate referents, the two handshapes investigated in this study – 1-handshape and 2-handshape (Figure 1) – tend to be found across many sign languages. For example, the extended 1-handshape prototypically represents ‘a long thin entity’ (Frishberg 1975, Wallin 1994) like a human being, but could also represent a snake or small scurrying animal, or even inanimate objects such as a lightning bolt. The bent 1-handshape, when referring to an animate referent, prototypically represents a human – or human-like creature, including anthropomorphic animals – with an arched back, or something small and thin like a worm, otherwise it can represent something inanimate, like a hook or a razor – see Wallin (1994), who argues that the bent 1-handshape is rarely used to represent humans in STS. An extended 2-handshape prototypically represents a human’s legs or some other legged animal (e.g. Brentari et al. 2013, Dudis 2004, Lepic and Occhino 2018, Schembri 2003, Supalla 1986, Taub and Galvan 2001, Wallin 1994, Zwitserlood 2003).

Thus, it seems as though both the 1- and 2-handshape (Figure 1) can be used to represent human(-like) and animal referents, but what conditions the choice of either handshape? Beal-Alvarez and Trussell (2015, 8–9) note that ASL signers can choose between a 1- or 2-handshape – or even involve the whole body in an enactment – to depict a human walking, but do not give any explanation for the factors influencing this choice. It has been reported that most sign languages have separate classifiers for animate referents (i.e. humans and other animals), but that the specific handshapes used to represent these referents differ across sign languages (Zwitserlood 2012). For example, the 1- and 2-handshapes in a vertical (upright) orientation are used to represent human referents in many Western sign languages, whereas several Asian sign languages – e.g. Hong Kong Sign Language, Taiwan Sign Language, and Thai Sign Language – use the Y-handshape (thumb and pinky extended) to represent a human (Zwitserlood 2012, 163).^[1] However, the 2-handshape can adopt various orientations, the 1-handshape is generally upright when representing a human(-like) referent. When the 2-handshape refers to animals, it is often bent (e.g. Wallin 1994, Zwitserlood 2012).

One of the most detailed analyses of factors and constraints in the choice of handshapes used for humans in depicting constructions comes from Engberg-Pedersen’s 1993 description of Danish Sign Language (DTS; Danish: dansk tegnsprog). Engberg-Pedersen (1993, 248–9) argues that the 1-handshape is used to express ‘approaching’, ‘passing’, and ‘leaving’ (from the perspective of the signer), as well as ‘moving along a route’, but is unlikely to be used to express the manner of a stative (‘sit/lie/stand’), ‘jumping’, or motion focusing on the legs moving. Conversely, the 2-handshape is used to express ‘motion focusing on the feet/legs’, the manner of a stative (‘sit/lie/stand’), ‘stand on one’s head’ (and other specific orientations), and motion that involves a change of location (e.g. ‘into the kitchen’), but is generally not used to express ‘approaching’ or ‘moving away from a group/human’ (Engberg-Pedersen 1993, 247–8).

Thus, in DTS, the straight 2-handshape is used to represent a human being’s legs or feet when standing or moving, which can be used to emphasise different orientations of the body. Meanwhile, the (slightly) bent 2-handshape is used in DTS when a person moves in a certain direction, to a goal, or within an area, thus emphasising a person’s change of location, rather than the specific mechanics of walking/running (Engberg-Pedersen 1993, 247). In conclusion, Engberg-Pedersen (1993, 250) argues that the choice of handshape in depicting constructions in DTS denoting humans – i.e. whether 1- or (straight or bent) 2-handshape – depends on which aspects of a motion event is highlighted.^[2]

For STS, the use of classifier-type handshapes in depicting constructions has been described previously (e.g. Bergman and Wallin 2003, Wallin 1994), but it remains unclear to what extent the animacy of a referent influences the choice of handshape in depicting constructions. According to Wallin (1994, 105–6), the 1-handshape is often used to represent human or human-like referents (e.g. trolls), but can also be used to represent a cat or dog, or even a snake if articulated with the index finger pointing in the direction of movement. Thus, similar to what has been found for other sign languages, the upright 1-handshape in STS is generally preferred for human(-like) referents or animals higher on the animacy scale (e.g. cats and dogs). However, the 2-handshape can be used for both humans and non-human animates (i.e. animals). Wallin (1994, 114–5) says that the straight 2-handshape is mainly used for human and human-like referents, but similar to DTS, the 2-handshape is more flexible than the 1-handshape in expressing different orientations of the referent – for example, whether standing up, lying down, or being upside-down. Wallin (1994, 117–8) notes that the orientation of a human(-like) referent depicted with the straight 2-handshape is ambiguous with regard to the direction of the referent’s face, which can either be on the palm-side or the back of the fingers. However, this is generally only the case for location depictions, and when the depicting construction denotes a motion, the depicted referent’s face is always on the back of the fingers, i.e. the side of the knuckles (Wallin 1994, 118). The bent 2-handshape, Wallin (1994, 118–20) explains, can be used for both (crouching) humans and other animate referents, whether two-legged (e.g. birds) or four-legged (e.g. cats, squirrels), and can be used to express both location and motion events.

Based on the previous work on STS as well as several other sign languages, it is clear that both the 1- and 2-handshapes are used in various constructions representing animate referents. Yet, it remains unclear which handshape STS signers choose in a depicting construction, and whether there are constraints on the choice of a certain handshape category (e.g. 1- vs 2-handshape), or on the form variations (e.g. straight vs bent) within it. In this article, we limit ourselves to exploring the factors underlying a signer’s choice of either the 1- or 2-handshape when referring to human(-like) and animal referents in depicting constructions in STS. Based on the previous research, we hypothesise that the 1-handshape is more likely to be used with human than animal referents and that the 2-handshape is more likely to be used with more complex motion/orientation and to be used with both human and animal referents. There are, of course, many referential expressions in sign language discourse (Ferrara et al. 2023, Keleş et al. 2023), but we focus on the depicting constructions only in this study, and more specifically on the handshape choices within them. The types and number of constructions found will also depend on the text type, with depicting constructions being more common in narrative texts in STS and other sign languages (Börstell et al. 2016a), which is why we also take the text type (Section 2.3) into account.

2.3 Narrative text and conversational text

A narrative text comes in different types and formats and can be used in many contexts and for various purposes. What is considered a narrative text ranges from an imaginary story told to a child at bedtime, to a personal retelling of what happened at the workplace yesterday, or a more intimate recounting of an emotional event shared privately (Herman 2009). The content in narrative texts varies drastically as the speakers of these narratives choose to focus on different themes, but also based on whether the narration is free or more directed to fit a special purpose.

As Ochs and Capps (2002) argue, a face-to-face retelling of wordless picture books constitutes a kind of prototype of a narrative activity. This is the type of text used in the narrative part of the data sample in our current study (Section 3), namely signed retellings of the picture books Frog, Where Are You? (Mayer 1969) and The Snowman (Briggs 1978). These picture book stories have been used in several linguistic research studies on topics related to language acquisition, narrative skills, and cross-linguistic comparisons of grammatical and discourse structure, in both spoken and signed languages. For sign languages, both stories have been used as elicitation material for corpus building, and especially Frog, Where Are You? (often referred to as “the frog story”) has been used in studies on many different sign languages (e.g. Engberg-Pedersen 2003, Ferrara et al. 2023, Gulamani et al. 2022, McIntire and Reilly 1996, Polinsky 2018, Sipronen and Kanto 2021). These narratives in the STS Corpus are structured in such a way that one signer in the dyad is assigned the task of retelling one of the two stories to the other signer (i.e. the addressee), who in turn mainly participates by providing backchanneling/feedback (for backchanneling in STS, see Börstell 2024, Mesch 2016). Thus, each dyad (signer pair) in the STS Corpus is represented for both stories, with one signer retelling one of the stories and the other signer retelling the other story.

The other text type used in this study concerns conversational texts elicited through prompts about self-experienced events. In these conversational texts, one signer at a time provides the main portion of the content, but sometimes with more active feedback engagement by the addressee, at times involving longer interspersed comments or even overtaking the conversational floor.

Specifically, the conversational texts sampled for this study are about the signers’ experiences of when they first encountered/learned sign language, which is why there may be more elaborate comments or turn-taking shifts between the two signers in a dyad if the addressee wants to comment on content or share their own related experiences as the conversation progresses. Ochs and Capps (2002) point out that a conversational narrative of this type is shaped and re-shaped turn by turn in the course of conversation, and in these circumstances, the conversational narration takes shape as both participants develop an understanding of the events presented. The comments and reactions from the addressee do not interfere with the speaker, but show attentive listening (e.g. Bavelas et al. 2000, Bavelas and Gerwing 2011, Goodwin 1986a,b). The events that a signer retells in these conversational narratives resemble small stories (brief narratives) and, like a prototypical narrative text, may also consist of, e.g. depicting constructions and forms of enactment, although seemingly less frequently than in a longer narrative (Börstell et al. 2016a). These short narratives are intended to be perceived only as a chain of separate, standalone narratives and contribute to part of a larger context or framework of conversation. In this article, we will refer to these conversational narrative texts simply as conversational texts, in contrast to the picture-story retellings, which are referred to as narrative texts. In addition to these two text types in the STS Corpus, there is a text type labelled presentation, which consists of the files in which the signers introduce themselves to each other. In terms of data size, this text type is by far the smallest of the three and is not included in our annotated subsample (see further details in Section 3).

Narratives in sign languages involve different ways of depicting referents and viewpoints of events. Previous research has shown how signers navigate among multiple events and perspectives (Perniss 2007), and, at the same time, give us information about the characters in a narrative text and what their actions are (e.g. Ahlgren et al. 1994, Bahan and Supalla 1995, Dudis 2004, Ferrara et al. 2023, Gulamani et al. 2022, Liddell 2003, Rathmann et al. 2007, Roy 1989). In a narrative text in sign language, the signer uses several devices to maintain consistency and continuity to mark where referents are in the signing space and simultaneously give information about the activities of the referents in the narrative and other relevant details. To do this, the signer uses visual markers to guide us through the narrative, such as the use of different types of depicting constructions and representation in the signing space, which Perniss (2007) calls spatial representation. She shows that a signer can exhibit spatial representation in the signing space in several ways. One way is where parts of the signer’s body are immersed in the characters of the narrative. In doing so, the signer is giving the characters’ actions, thoughts, and dialogues from a first-person perspective. This space is life-sized, and the signer utilises depicting constructions and constructed action from a first-person perspective, referred to here as the character’s perspective (Cormier et al. 2013a). Another way or use of topographical spatial representation space is when the signer adopts an external perspective on the narrative events, rather than being immersed in a specific character, which means the signer is narrating the events from a narrator’s perspective, presenting the events from a third-person perspective, referred here as the observer’s perspective (Cormier et al. 2013a). Unlike the life-sized first-person perspective, this spatial representation is reduced in scale and also incorporates depicting constructions (e.g. Cormier et al. 2013b, Dudis 2004, Liddell 2003, Perniss 2007, Schembri 2003). The purpose of this article is not to explore a signer’s use of perspectives in narrative texts; rather, we discuss a signer’s perspective when using different forms of depicting constructions with 1- or 2-handshape. In this study, we use character perspective and according to Perniss (2007) it is when a signer is immersed in a character and “is conceptualised as being within the event space” (Perniss 2007, 1316). When the signer “is outside of the event conceptually, viewing the scene from the perspective of an external observer” (Perniss 2007, 1316), we call it an observer perspective.

This narrative practice entails that signers use “iconic forms,” which are considered a narrative ideal, although not all signers are necessarily renowned storytellers (Engberg-Pedersen 1999). Despite this narrative ideal, Beal-Alvarez and Trussell (2015) analysed the use of depicting constructions in adults’ narrative retellings of a storybook in ASL and found that all of the signers retold the main events in similar ways, but varied in how much they used depicting constructions and forms of enactment to represent different characters in the narrative. In this study, we will not explore the type of enactment usually referred to as constructed action (Beal-Alvarez and Trussell 2015, Cormier et al. 2013b, Puupponen et al. 2022), in which a signer depicts a character’s actions/movements/expressions with their whole body (see also Dudis 2004, Liddell and Metzger 1998, Taub and Galvan 2001). Instead, the focus is on depicting constructions in which a signer expresses a referent’s location/movement in a narrative scene through the use of a whole-entity handshape (Engberg-Pedersen 2010, Schembri 2003), whether or not the rest of the body is involved in some way (cf. Dudis 2004) – see also Section 3.2.

3.1 The STS Corpus data

The STS Corpus data were collected in the years 2009–2012, as part of a corpus-building project (Mesch et al. 2012a, Mesch 2015). The full STS Corpus consists of 298 files, each a filmed, dyadic interaction between signers engaged in conversation, seated opposite to each other in a studio setting with multiple video cameras around. The corpus has been annotated in full for all manual signs and additionally contains written, idiomatic Swedish translations of the conversations. In total, the STS Corpus comprises 189,679 sign tokens across 42 signers of different ages and regional backgrounds, spanning around 24 h of video-recorded STS signing. As mentioned in Section 2.3, the STS Corpus consists of three text types: conversational (76.2% of the total tokens), narrative (18.8%), and presentation (5%).

For our detailed analysis, we subsampled a part of the STS Corpus for additional manual annotation and subsequent analysis. The subset comprised 45 out of the 298 annotation files in the STS Corpus, and can be split into two parts based on the text type. This subset was sampled based on two main criteria: first, we needed to limit the number of files to make manual annotation feasible within the scope of the project; second, we wanted to maximise the diversity of signers and text types. Thus, we sampled 15 pairs of signers (out of 21 pairs total in the STS Corpus) and selected one conversational and two narrative files from each pair. This is because the conversational files generally contain sign production from both signers in the pair, whereas the narrative files mostly contain signing from the signer retelling the story. The conversational set consists of 15 files, which all address the topic “How did you first encounter/learn sign language?”. The narrative set consists of 30 files, 15 of which are retellings of the wordless picture storybook Frog, Where Are You? (Mayer 1969) and 15 are retellings of the wordless picture storybook The Snowman (Briggs 1978). The total 45 files annotated thus cover 30 of the 42 signers in the STS Corpus and comprise 26,317 sign glosses (14% of the 189,679 sign tokens in total in the STS Corpus), corresponding to about 3.5 h of annotated video data (1 h 45 min of conversational data; 1 h 43 min of narrative data).

While both text types are prompted, the conversational texts are more free and variable than the narrative texts in terms of content, being driven by the experiences unique to the interlocutors rather than the topics constrained by the contents of the picture stories. The same dyads (30 signers total) are represented in both conversational and narrative texts. For each dyad of participants – pairings are never scrambled across files in the STS Corpus – both signers contribute actively to the conversational task, whereas each signer was assigned one of the two stories for the narrative task, each of which is consequently more one-sided in terms of sign production. Thus, 15 signers provided a retelling of Frog, Where Are You? and the other 15 signers a retelling of The Snowman. Both stories are presented as wordless picture stories in book form (see Appendix A for story summaries and lists of referents). Frog, Where Are You? contains 26 picture frames and revolves around a little boy and his dog, who go out in search of the boy’s escaped pet frog (Mayer 1969). The Snowman contains 167 picture frames and tells the story of a friendship between a boy and a snowman who comes alive, sharing a magical adventure together before the snowman melts the following day. During the recording of the narrative tasks, the stimulus material (i.e. the books) was presented to the designated signer ahead of recording, but was accessible during the filming in case the signer forgot crucial parts of their assigned story (Mesch 2015, Mesch et al. 2012a).

The 30 signers (14 women; 16 men) in the corpus subset analysed in this study range from 20 to 72 years old at the time of recording (mean age: 43.5). All signers identified STS as their first language and the age of acquisition of STS ranged from 0 to 7 years. Furthermore, 5 have deaf parents, and 25 have hearing parents. Importantly, all signers are proficient in STS and use it as their primary language.

3.2 Data annotation

As mentioned in the previous section, the data used for this study consist of two parts: first, the entire STS Corpus data are used for a general exploration of sign distributions; second, a subset of 45 corpus files was sampled for a more in-depth analysis. For this second part, 45 annotation files, subsampled from the STS Corpus, were annotated with the ELAN software^[3] (Wittenburg et al. 2006) to add annotations specifically targeting the goals of this study. These annotations follow the general coding scheme outlined in the annotation guidelines developed for the broader project Whole-entity classifiers in sign languages: a multi-perspective approach, and are annotated using the associated ELAN template.^[4] The annotation procedure for our subset is outlined in the following.

First, a meticulous examination to visually detect and inspect every instance of potential depicting constructions with the 1- and 2-handshapes (including bent and moving fingers) across all 45 annotation files from the STS Corpus subset was carried out. Each annotation file is associated with two (main) video files played synchronously, each showing one of the two signers in the dyad. This visual inspection consisted of watching each conversation/narrative in the ELAN interface and adding annotations for every target occurrence – that is, any potential depicting constructions with the 1- and 2-handshapes referring to animate referents – following the annotation guidelines. This annotation task was carried out by a native signer of STS (the first author), with difficult cases discussed with the second author to reach agreement on the categorisations and definitions.

More specifically, the target signs were any sign using the 1- or 2-handshape to refer to human(-like) or animal referents using either the (upright) 1-handshape as a whole-entity depiction, or the 2-handshape with the two fingers depicting legs. Thus, a 2-handshape with the fingers pointing upwards, referring to two upright entities (for instance, two humans walking together) is not included. Such cases are excluded to restrict the analysis to the 1- and 2-handshape forms specifically for cases in which the 1-handshape represents a single (upright) animate referent or the 2-handshape represents the legs of an animate referent. However, two-handed forms with either the 1- or 2-handshape, in which each hand represents a single referent, are still included.

For each target depicting construction in the subset, additional annotations were coded on separate tiers relevant to this study.

The handshape tier was used to annotate the construction’s handshape, which could include various handshape variants within the 1- and 2-handshape categories (e.g. bent/straight and still/moving fingers).

The congruence tier was used to annotate the construction’s form relative to possible lexical signs, focusing on whether the hand configuration, movement, and path are congruent (resembles that of a lexical sign) or depictive (more complex). Given the purpose of this study, we wanted to exclude instances of lexical signs. However, in some instances, it was a challenge determining whether a form should be classified as a depicting construction, because the form was more or less congruent with a lexical sign. Furthermore, in some cases, a sign was determined to be a depicting sign in our annotation, despite being annotated as a lexical sign in the original corpus annotations. Our main criterion was whether the 1- or 2-handshape was used to iconically represent the motion or location of an animate referent in the depiction of an event. While many of these occurrences are clearly depicting, involving a complex movement and interactions with other locations or entities (e.g. the other hand), some turned out to be less clearly depicting, particularly in combination with another verb-type sign – see further discussion on this issue in Section 5. Thus, we could use an aggregation of information from other tiers (described in the following) to establish whether a sign should be labelled depicting.

The tier movement concerned the construction’s movement, whether straight, circular, arc, or complex (e.g. a combination of two or more movements). This was done to determine whether a specific handshape exhibited a preference for a particular movement, i.e. had a tendency for a certain type of movement.

The tier direction included information about the construction’s general direction in signing space, if the depicting sign was moving forward, left, right, or away or towards the signer.^[5] However, we did not use CLP direction in our analysis, but it was annotated as part of the project within which this study was conducted.

The tier interaction provided details about possible other referents depicted in the construction, represented either by the other hand or by the signer’s body.

The tier second-hand included information about the animacy of any referents represented by the other hand, with possible values including human (including human-like referents), animate, and inanimate.

The mouthing tier included information only if there was a simultaneous mouthing of a Swedish word.

The tier event was used to annotate whether the movement signified the motion or location of the referent.

The meaning tier was used to assign a label approximating the construction’s meaning, such as ‘walk’ or ‘fall’.

Finally, the comments tier was used to add additional information. This tier was used to note whether a depicting construction consisted of two hands, and potentially the specific referent depicted – for example, if one hand referred to the boy and the other to the snowman. Sometimes, two hands can refer to one referent, e.g. a deer. Here, we also comment on whether a sign seemed to have a grammatical function rather than a depicting construction (see discussion in Section 4.3).

3.3 Data processing

The STS Corpus annotation files (Mesch et al. 2012b) were downloaded from The Language Archive.^[6] The sign annotations from both the STS Corpus, as well as the dedicated annotations added to the files in our subset, were extracted from the annotation files and further processed, analysed, and visualised with R v4.4.1 (R Core Team 2024) and the packages ggtext v0.1.2 (Wilke and Wiernik 2022), glue v1.7.0 (Hester and Bryan 2024), here v1.0.1 (Müller 2020), paletteer v1.6.0 (Hvitfeldt 2021), patchwork v1.2.0 (Pedersen 2024), scales v1.3.0 (Wickham et al. 2023), signglossR v2.2.6 (Börstell 2022), tidyverse v2.0.0 (Wickham et al. 2019), and tinytable v0.4.0 (Arel-Bundock 2024).

In our subset files, 1,114 signs were originally annotated according to the categories listed in Section 3.2 when using broad inclusion criteria of looking at any form using the target 1- and 2-handshapes that refer to animate referents. However, only 864 (77.6%) of these annotations were deemed to be depicting signs. Some of the signs in this final set include signs that are not annotated with the ‘@p’ tag used for marking depicting signs in the STS Corpus. For example, the 1-handshape motion verb often used with the meaning ‘approach’ was initially annotated as a depicting sign in previous releases of the STS Corpus, but later re-categorised as a lexicalised sign in more recent releases of the corpus, following updated glossing conventions. In our annotation procedure, we could not reliably distinguish these forms from other depicting signs in the subset data, and thus decided to annotate them as depicting signs much like in the earlier releases of the STS Corpus, thereby overriding the sign gloss annotations.

In the following, we will thus report the results of depicting constructions among the 189,679 sign tokens in the entire STS Corpus as part of our full corpus data set, as well as the 864 depicting constructions annotated in more detail in the subset. While all 30 narrative files contain depicting constructions with our target handshapes (i.e. the final 864 annotations), four of the conversational files did not contain any target handshape depicting constructions, leaving 11 out of 15 sampled conversational files as part of the results (Section 4). When describing the general handshape categories, we will refer to the two categories of target handshapes as 1-handshape and 2-handshape, respectively, with each category consisting of a variant with straight and bent fingers. Additionally, we do not make any distinction between handshapes with the thumb out or tucked, nor between 2-handshape variants with the two fingers spread or in contact side-by-side.

The data and code for this study can be found at: https://osf.io/h7cx9/.

4.1 Depicting constructions in the STS Corpus

As outlined in Section 3.1, the STS Corpus comprises 189,679 sign tokens, across different text types labelled as conversational (76.2% of the tokens), (elicited) narrative (18.8%), and presentation (5%). While most tokens in the corpus come from conversational-type texts, depicting constructions of all types as defined by the ‘@p’ tag in the STS Corpus ( n = 5,949 ; 3.1% of the total tokens) are relatively more frequent in the narrative texts, where depicting constructions account for 10.5% of the total tokens, compared to 1.5% in the conversational texts and <0.4% in the presentations – see also Börstell et al. (2016a) for lexical frequency statistics of the STS Corpus.

When looking at the handshapes used in depicting constructions in the STS Corpus as a whole, there are 55 unique handshape labels among all constructions annotated as depicting constructions. Among these 55 handshapes, the flat hand is the most commonly used handshape, found in 1,125 (17.6%) of all occurrences of depicting constructions. Figure 2 shows the 15 most frequent handshapes found among depicting constructions in the STS Corpus, distributed across one- and two-handed constructions. Figure 2 also shows that four of the 15 most frequent handshapes belong to our target 1- and 2-handshape categories (highlighted by arrows).

Figure 2

The distribution of the 15 most frequent handshapes within depicting constructions within the STS Corpus, and their distribution across one- and two-handed constructions. Target handshapes in this list are highlighted by arrows.

Out of all handshapes identified in the glosses of depicting signs in the STS Corpus, nine belong to our target handshapes, including all variations of thumb involvement and bent, straight or spread fingers, and when collapsing one- and two-handed constructions. These nine handshapes account for 1,296 tokens, which is 20.3% of the total number of depicting constructions in the STS Corpus. Of these target form occurrences, 553 (42.7%) correspond to the 1-handshape category, and 743 (57.3%) to the 2-handshape category.

When looking at the number of hands, the overall distribution of one- vs two-handed depicting constructions in the STS Corpus is fairly balanced, with 3,500 occurrences being one-handed (54.7%) and 2,899 being two-handed (45.3%) – see Börstell et al. (2016b) and Östling et al. (2018) for distributions of one- and two-handed signs across sign languages.

In the depicting constructions using one of our target handshapes, a general event type with regard to whether the sign expresses motion or location can be induced from the sign gloss (i.e. its description in Swedish) in 1,116 occurrences (86.1% of the total target handshape occurrences). Of these 1,116 depicting constructions, most are expressing a motion event ( n = 952 ; 85.3%) and the remaining a location event ( n = 164 ; 14.7%), and among all these event-identifiable constructions, the 2-handshape depicting constructions ( n = 728 ; 65.2%) are about twice as frequent as the 1-handshape depicting constructions ( n = 388 ; 34.8%). However, the relative distribution of motion vs location events for either handshape category is more or less the same, with around 14–15% of the depicting constructions expressing a location (1-handshape: n = 54 ; 13.9%; 2-handshape: n = 110 ; 15.1%) and 85–86% of the depicting constructions expressing motion (1-handshape: n = 334 ; 86.1%; 2-handshape: n = 618 ; 84.9%) (Figure 3).

Figure 3

Distribution of 1- vs 2-handshapes in depicting constructions by event type in the STS Corpus; absolute numbers (top graph) and percentages (bottom graph).

4.2 Depicting constructions in the corpus subset

As in the STS Corpus in general (Section 4.1), the depicting constructions in the subset are heavily skewed towards the narrative texts compared to conversational texts, despite the conversational text annotation files containing slightly more of the total sign tokens in the subset corpus files (57.7%). Figure 4 illustrates that the split between 1- vs 2-handshapes exhibits quite different distributions in the two text types, with 1-handshape constructions being slightly more frequent than 2-handshape ones in the conversational texts, but the 2-handshape constructions being overwhelmingly more frequent than the 1-handshape ones in the narrative texts. However, the occurrences of depicting constructions with a 1- or 2-handshape in the conversational texts are too few to warrant the conversational occurrences being further analysed on their own. Therefore, we will in the following report aggregated numbers across both conversational and narrative texts when exploring factors underlying the choice of handshape.

Figure 4

Distribution of 1- vs 2-handshapes in depicting constructions by text type in the corpus subset.

With regard to event type, the subset and target handshape constructions pattern much like the depicting constructions in the STS Corpus as a whole, in that most depicting constructions with our target handshapes express motion rather than location (Figure 5). There is a clear difference in the distribution of bent vs straight fingers in the handshape categories, with the 2-handshape being much more likely to use bent fingers regardless of event type (see the light blue colour in Figure 5). Figure 5 also shows that the 1-handshape is more likely to use straight than the bent fingers, regardless of event type.

Figure 5

Distribution of 1- vs 2-handshapes in depicting constructions by event type in the corpus subset.

Figure 6 shows the referent animacy in the 520 depicting constructions for which the referent denoted could be established, and the referent being either human or human-like (i.e. a snowman) or an animal – see Appendix A for more details about referents in the narratives. The majority of these 520 depicting constructions ( n = 388 ; 74.6%) referred to a human(-like) referent, and a smaller number ( n = 132 ; 25.4%) referred to an animal. As Figure 6 shows, the 1-handshape with a straight finger is much more likely to represent a human referent than an animal referent. Furthermore, the bent 2-handshape is used more frequently than a straight 1-handshape when referring to an animal referent. The 2-handshape is used by signers relatively more frequently, regardless of whether it refers to a human or an animal, and it is also much more likely that the 2-handshape occurs with bent fingers than the occurrence of the bent 1-handshape. We will further discuss the frequent occurrence of the bent 2-handshape in the data in Section 4.3.

Figure 6

Distribution of 1- vs 2-handshapes in depicting constructions by referent animacy in the corpus subset.

Turning to meanings expressed with our target handshapes, Figure 7 shows the distribution across the 1- and 2-handshapes. Figure 7 illustrates that the depicting constructions with a 1-handshape primarily express a more general motion, such as ‘walk’, ‘approach’, ‘leave,’ or ‘pass by’, whereas the 2-handshape constructions denote a wider range of meanings. Notably, the 2-handshape is used for motion events like ‘jump’, ‘fall’, ‘land’, and ‘climb’, as well as for postural meanings of location, such as ‘lie’ and ‘sleep’. This aligns with the 2-handshape being preferred with meanings involving the legs/feet, whether to emphasise their movement or to express the orientation of the body (e.g. not being in a default upright orientation).

Figure 7

Distribution of 1- vs 2-handshapes in depicting constructions by meaning in the corpus subset.

The types of movement of the depicting constructions observed in the subset are detailed in Figure 8. This figure reveals that while both 1- and 2-handshapes frequently employ straight or arc movements, the still (i.e. stationary hold) articulation is predominantly associated with the 2-handshape. This is specifically associated with meanings, such as ‘lie/sleep’ and ‘sit/stand’, as shown in Figure 7. The 2-handshape is used with diverse movement types, with the arc and circular movement types being relatively much more frequent than in the 1-handshape category.

Figure 8

Distribution of 1- vs 2-handshapes in depicting constructions by movement type in the corpus subset.

Looking at the distribution of one- vs two-handed depicting constructions by handshape in Figure 9, we can observe that it is more common for the 2-handshape than the 1-handshape to be used in a two-handed construction in which the two hands have the same handshape, although both categories occur in more one-handed constructions overall. Since two-handed construction may involve different references on the two hands, we wanted to look in more detail at the referents represented by the hand(s) in one- vs two-handed depicting constructions. Table 1 shows the distribution of referents across one- and two-handed constructions when the construction as a whole refers to a single referent or a uniform group of the same kind (in this case, multiple frogs). What we can read from Table 1 is that the human(-like) referents – ‘boy’ and ‘snowman’ – overwhelmingly occur in one-handed constructions. In the few exceptions, where they as singular referents are depicted in a two-handed construction, the reference to the second-hand seems unclear. However, an inspection of these specific cases shows that they are cases of either fragments or assimilation. Fragments are residual traces from a previous construction, e.g. when the signer has previously used two 1-handshape hands where one hand is referring to the boy and the other to the snowman, and that second-hand remains across a subsequent construction. Assimilation can happen due to either phonological symmetry across both hands (the second-hand mirroring some form) where the second-hand does not add meaning, or through contextual co-articulation due to a preceding/following sign/construction being two-handed. Notably, we can see that the signers generally use a one-handed depicting construction when referring to the snowman or the boy. Similarly, in reference to one animal rather than a group of frogs, signs about the frog are also one-handed despite it being a four-legged animal – that is, no two-handed 2-handshape constructions referring to a single frog. On the other hand, the more saliently four-legged referents – ‘dog’ and ‘deer’ – favour such a two-handed depicting form, thus representing the four legs of the animal with two hands. When the referent is multiple frogs (i.e. the frog family), the preference is a two-handed form, thus using the two hands to represent multiple individuated referents (Börstell In press).

Figure 9

Distribution of one- and two-handed constructions by handshape in the corpus subset.

4.3 Constructions involving motion verbs

While annotating the data, two types of multi-sign constructions involving our target depicting constructions stood out in the sense that they appeared to have less of a depicting role than other depicting constructions. First, a construction in which the bent 2-handshape denotes a movement to a location at which another, more central, action takes place. Second, a construction in which a straight horizontal 1-handshape denotes a movement into a defined space. In the latter construction, the movement of the 1-handshape does not specify the manner, but rather the transition from one space into another space, such as from an outdoor space to an indoor space, or from one room of a house into another. In the following, we describe the nature of the two constructions, with examples from the subset and the STS Corpus as a whole, to illustrate their function and meaning.

With regard to the first form with the bent 2-handshape, we found that it often occurred in a construction together with another verb directly following it, thus forming a sequence reminiscent of a serial verb construction (cf. Bos 2016, Couvee and Pfau 2018). Here, the first verb is the bent 2-handshape, where the hand moves along a simple linear path (horizontally or vertically), expressing a general movement towards a location at which some specified action takes place. Thus, the motion verb expresses a general change of location necessary for another action to happen (e.g. fetching an object from some other location), rather than emphasising the motion event in itself. We can see examples of this in our data in Examples (1)–(2), where the motion verb mainly expresses that the agent had to change locations due to some other referent being located there or the action being tied to a specific location. The articulation of the motion verb in such constructions is rapid and short, often accompanied by closed lips. The second verb, which then appears after the motion verb – e.g. FETCH in (1) and SAY-TO in (2) – contributes more to the overall meaning of the construction as a whole. We interpret this as the motion verb having a weaker depicting function, since the manner and path of the motion event are de-emphasised in favour of simply expressing a change of location. Out of 99 tokens annotated with the MOVE(2bent) sign in our subset data, 46 (46.5%) occur in a context interpreted as this de-emphasised motion construction, distributed across eight different signers, which points to a broader pattern than a restricted or idiosyncratic use.

This type of use seems similar to what Engberg-Pedersen (1993, 247) described for DTS (Section 2.1), in which the bent 2-handshape is used to express a change of location without emphasising the mechanics of the motion itself. That is, there is less emphasis on whether someone is walking or running, and in which manner. Because of this, we decided to also look at the distribution of word classes directly following the target depicting constructions. To do this, we included any sign following a target construction, if it started within 500 ms of the end of the target, otherwise we categorise it as being followed by an end (marked ‘[ . ]’). The word classes are taken directly from the STS Corpus, which is fully tagged for word classes on the sign-type level (Östling et al. 2015). Figure 10 shows the results of the word class context, demonstrating that the majority of signs directly following our target depicting constructions are categorised as verbs, but there is a similar distribution between the 1- and 2-handshapes overall. For the MOVE(2bent) sign specifically, as many as 47 (47.5%) of the occurrences are followed by a verb, 38 (38.4%) by another word class, and 14 (14.1%) are not immediately followed by a sign. However, whereas we find many examples of the type of construction illustrated in Examples (1)–(2), we do not find many examples of an equivalent construction with the 1-handshape in our data – that is, a construction in which the 1-handshape expresses a general motion as a change of location, followed by a ‘main’ verb. In fact, we barely find any example with the 1-handshape that corresponds to the [MOVE(2bent) + VERB] in our subset. We do find an example that has both 1- and 2-handshapes in the same sequence, but then the 2-handshape sign is the one immediately preceding the second (main) verb OPEN – see Example (3).

Figure 10

Distribution of word class following target depicting construction in the corpus subset.

The one example we find in the subset that uses the 1-handshape in a similar construction is more about a person approaching a location or situation with other referents, described rather from an observer’s perspective (third-person perspective or observer-like viewpoint) – see Example (4).^[7]

The meaning associated with this use of the 1-handshape is ‘go there’, specifically how the individual has changed location without specifying the manner of movement. However, looking at the larger STS Corpus, we find Example (5), which is similar to Example (4) in its use of the 1-handshape motion verb together with another ‘main’ verb, but one difference is that in Example (5) the hand moves from one side to the other in front of the signer. When comparing these to the 2-handshape construction (with bent fingers), we find it plausible based on our interpretation that the motion verb in [MOVE(2bent) + VERB] expresses a change of location when the perspective is with the referent, i.e. a character’s perspective who is the focus of the event(s), whereas [MOVE(1) + VERB] expresses a change of location when the referent is brought in or observed from outside the main location of events, i.e. the observer’s perspective. That is, [MOVE(2bent) + VERB] corresponds to ‘go to do X‘ and [MOVE(1) + VERB] corresponds to ‘(outsider) arrives to do X‘ or ‘moves over there to do X’.

We have not seen the construction [MOVE(2bent) + VERB] being used for animals in our analysis of the data in this study, which may indicate that it might only be used for human referents. However, if the character’s perspective (that of a human character) is a key feature of the construction, we do not have adequate data to answer the question, as none of the narratives in the dataset feature an animal as the main character.

We consider [MOVE(2bent) + VERB] to be similar to serial verb constructions, which have been found across both spoken (Aikhenvald and Dixon 2005) and signed languages (Couvee and Pfau 2018), but as morphological marking is not as well-defined for signed languages as it is for spoken languages, we cannot apply some of the criteria often used to define serial verb constructions. For example, it remains unclear whether the movement modification (the speed) in the first verb MOVE(2bent) constitutes a form of morphological marking. Additionally, the second verb following the MOVE(2bent) describes the main event rather than, for example, indicating the result of the movement expressed in the first verb. This is because MOVE(2bent) seems to express solely a change of location without specifying the manner of movement. As this article does not focus on serial verb constructions in STS, the topic is not discussed further here. Furthermore, it may be premature to suggest that it is a grammaticalised construction in STS, since the [MOVE(2bent) + VERB] construction is also similar to types of pseudo-coordination of motion verbs in other languages. For example, in Swedish, a pseudo-coordination construction with a motion verb often renders the motion verb itself less important with regard to the motion or movement, focusing instead on the verb that comes after. As Kvist Darnell (2008, 160) argues with a construction like Example (6), the motion verb gå ‘to walk’ is a means for the second verb, handla ‘to shop’, to take place – i.e. the activity/mechanics of walking is not important, but the motion verb only expresses that the main activity happens elsewhere.

The other construction we observed as less depicting involves the straight 1-handshape, pointing forward and held in a horizontal position, glossed here as ENTER(1), followed by a string of signs which tend to be either depicting constructions or some constructed action, expressed from a character’s perspective (first-person perspective or character viewpoint) – see Example (7).

The meaning of the construction ENTER(1) involves a change of location into a new, enclosed space where a more central event occurs. In our data, ENTER(1) frequently appears in retellings of The Snowman story, marking transitions as the Snowman moves between rooms in the boy’s house. ENTER(1) functions here to indicate entry into a new space where subsequent key activities unfold. In Example (7), ENTER(1) is presented from a character’s perspective (first-person perspective) and is followed by the lexical sign EXCITING (which is a feeling the character has when entering the enclosed space and is expressed from a character’s perspective), as well as the sign constructions WALK(2b) and LOOK-AROUND, all of which convey the character’s perceptions, thoughts, comments and actions within this newly entered space.

In Example (7), ENTER(1) is preceded by the lexical sign GO-IN(1) and accompanied by the mouthing of the Swedish word in (in(to)’). Like ENTER(1), GO-IN(1) involves a straight horizontal index finger (palm facing down) moving forward in a horizontal downward arc. The lexical sign GO-IN(1) exhibits a more rapid and reduced movement compared to the construction ENTER(1), where the arc movement is articulated with a bit more weight and is accompanied by a closed mouth. While both signs denote entry into an enclosed space, the lexical sign GO-IN(1) often precedes ENTER(1), and when the signer performs ENTER(1) in Example (7), he shifts his gaze direction and expresses immersion from a character’s perspective, a form of a narrative strategy. In Example (7), the construction with the lexical GO-IN(1) followed by ENTER(1) has a structure that is similar to what has been described for verb sandwiches (e.g. Bø 2010, Engberg-Pedersen 1993, Fischer and Wynne 1990).

We will not further discuss ENTER(1) as it is not the focus of this article. However, it is unclear what the 1-handshape in ENTER(1) refers to and whether it is a depicting construction, directional verb, or a fixed narrative strategy to express a change of scene. We note that the function of the index finger in the construction ENTER(1) remains unclear to us. While it undoubtedly is used for the motion of a human (or human-like) referent in general, it is unclear whether or not it should be categorised as a depicting handshape for a human(-like) referent. In a more clearly depicted case, the handshape is more likely to be a 1-handshape in an upright vertical orientation, with a downward arc movement. However, we interpret ENTER(1) as being on a gradient scale from depicting to more abstract, similar to MOVE(2bent), in the constructions in which it precedes a verb.