Scoping review of research methodologies across language studies with deaf and hard-of-hearing multilingual learners

4.1 Sampling and inclusion criteria

In order to achieve a comprehensive sample of original research papers on experiences of language acquisition, language development, language use, and language education of DML, we invoked a string of sampling procedures. First, we drew up a list of all studies known to the authors and their working groups. Following this, we enacted a snowballing system based on the authors referenced in these texts. At the same time, we carried out a search of peer-reviewed papers using the keywords “multilingual” (and variants such as “bilingual” and “plurilingual”) and “deaf” (as well as variants such as “DHH”) on the ERIC database and Google Scholar. Finally, we sent a list of papers found through these methods to colleagues conducting research with DML with the request that they add any additional original research papers of which they were aware. This method of sampling resulted in a list of approximately 80 papers across an unrestricted timeframe, which was then reduced according to the criteria set for inclusion. This method has various caveats, the most obvious being that it is entirely possible that we missed relevant studies, especially those published in languages other than English and German, the working languages of the authors.

For final inclusion, papers were judged on the following four inclusion criteria: (a) target population discussed, (b) aspect focused on in the study, (c) type of publication, and (d) publication venue. Any publications that did not meet these criteria were excluded. First (a), we included only studies examining DMLs. Second, (b) only studies whose main focus was on language(s) acquisition, language development, language use, or language education were considered. Thus, studies focusing on, for example, lived experiences in the community as a deaf immigrant, or describing general educational trajectories of subsamples of DMLs, were excluded from the review. Third (c), only original research papers were included. Finally (d), only studies published in peer-reviewed journals, peer-reviewed, edited books, and those published as Master’s or doctoral theses were included. This aimed to ensure that accepted standards of empirical research were adhered to. We included academic theses because these report original research often broader in scope than single-article papers, and are subjected to a rigorous review by the student’s research committee, but often remain unpublished and thus generally inaccessible. This process resulted in a total number of 33 original research studies for the review.

4.2 Variables included for review

The specific variables of interest for the methods review were developed using a recursive process of content-based and data-based considerations. The final aspects considered were (a) research questions posed, (b) population under consideration, (c) languages and cultures involved, (d) the research participants involved, (e) sampling methods, and (f) research design. We chose, for the purpose of this review, not to discuss study results. Since our goal was to present research methods amongst the population of DML, we look instead only to researchers’ choices when focusing on this group and its multilingual experiences.

5.1 Research questions

Studies reviewed examined a variety of research questions, including (a) families’ perspectives, choice and decision-making processes, (b) experience in the education system for both families and learners, (c) accommodations and strategies when working with learners, (d) service provider experiences and perspectives on educating learners, and (e) characteristics of DML (Table 1):

Families’ perspectives, choice and decision-making processes were examined by four studies. Research questions included those about navigating cultures and how this impacts family engagement (Batamula 2016), family language choice and its impact on communication (de Garcia 1993), how parents communicate in the home and community settings (Alfano 2019), and the parental decision-making process for those from culturally and linguistically diverse backgrounds (Steinberg et al. 2003). Only one study examined experiences in the education system for both families and DML with a retrospective view at how they navigated school systems, received accommodations and strategies to support language development, and how these supports impacted the academic outcomes of one DML (Baker and Scott 2016).

The accommodations and strategies examined across research questions in the studies targeted increasing DMLs’ language and/or literacy skills. One study considered understanding of grammar structures in reading comprehension (Cannon et al. 2020). Three studies focused on increasing vocabulary acquisition for DMLs. Cannon et al. (2010) utilized a pre-teaching and chaining intervention that incorporated visual and bilingual strategies to increase targeted, functional, ASL vocabulary, which was replicated by Guardino et al. (2014) with an older population of DML. Granda (2014) also investigated strategies to increase vocabulary (visual, repeated readings, culturally relevant pictures books, ASL-English bilingual strategies) and how often teachers used them. Kwon et al. (2022a) questioned whether a DML with Autism Spectrum Disorder (ASD) in a general education setting would increase their social interactions with classroom peers through social stories that incorporated ASL. Another group of three studies examined DMLs who use cochlear implants and looked at their speech development (Thomas et al. 2008; Waltzman et al. 2003), or tried to determine the languages each DML was exposed to at home and at school (Teschendorf et al. 2010). Four studies contained research questions about the service providers’ experience and perspectives on educating DML (Becker and Bowen 2018; Begon 2021), with two focused on interpreters’ work (Fishbeck 2018; Olsen 2018).

Just over a third of the studies (12 out of 33) examined characteristics of DML – understandable, given that this population has not been heavily researched, and that describing the characteristics of the learners forms a foundation for case studies, the starting point for any research about a relatively unknown population. Becker and Juche (2018) examined the characteristics of learners who are refugees, their strengths and challenges, and the resources required to support them. While Lummer (2011), Shin (2003), and Crowe and Guiberson (2021) examined lived experiences, Guiberson (2014) turned to the L1 and L2 (second language) proficiency, with a focus on the sources of the L2 exposure from parent survey and rating scales. Sivunen (2019) also considered the experiences of DML but focused on asylum seekers, while Youngs (2010) examined refugee camp stakeholder experiences, obstacles, and strategies for support. Only one study included a research question on language proficiency of DML who experienced language delays (Schrötzlmair et al. 2012), while a further four focused on background characteristics of DML. Of these, two investigated demographics and effective strategies to support learners (Lummer 1999; Swanwick et al. 2021), while Willoughby (2008, 2012 considered factors of L1 and L2 acquisition (i.e., becoming bilingual), which included family decisions to use/not use their heritage language, language choice and cultural heritage, family experience within the education system, professionals’ experience supporting DMLs and their families, and changes that are needed for further accessibility.

5.2 Population under consideration

We analyzed which characteristics of DMLs were reported in the studies. The results showed, first, that much of the research was completed with professionals about their experiences; only 19 out of the 33 studies reviewed included DML as research participants. As well, many posed research questions regarding particular subpopulations. These included learners who use cochlear implants, refugees, or learners from specific cultural backgrounds (Table 2):

We first examined the age and educational stage of those participants in studies who were the actual learners themselves. Only nine of the 19 studies reviewed noted the age or educational stage of the learners, while two further studies named the age learners began using a cochlear implant. One of the nine studies focused on learners of preschool age (3–4 years old), four on learners 5–12 years old, two on learners 14–22 years old, and two on adults (talking about their language learning at younger ages).

Next, we examined whether the studies reported the age of the IDMLs at the time their family immigrated to the country in which the study took place. Only six out of the eleven studies that included IDMLs reported the age at immigration.

Nine studies reported the hearing levels of the participants, with two exceptions reporting information about the learners’ cochlear implants (e.g., age received, length of usage), but not specific levels of hearing loss (Thomas et al. 2008; Waltzman et al. 2003). The remaining studies also reported the learner’s access to assistive hearing technology, yet the majority did not report on how long the learner utilized the auditory equipment within their previous and current schooling, or during their home life.

5.3 Languages and cultures involved

We examined whether the language(s) of the school, the home, and communities involved were reported. We found that studies examined learners, families, and professionals who utilized 35 spoken languages and 9 sign languages (Table 3; see Table 3a, Supplementary Material, for a full listing of languages involved):

Across the variety of spoken languages, following English, learners who were exposed to Spanish were studied most often (11 studies). Learners also used Arabic (six studies), French (four studies), Korean and German (three studies each), and Cantonese, Mandarin, and Russian (two studies each). The most common sign language used by participants was ASL (12 studies), followed by Australian Sign Language (two studies), and then German, Kenyan, Peruvian, Flemish, British, Danish, and International Sign Language (one study each). Two studies mentioned home signs and one mentioned an unspecified “native” sign language.

Just over half (17) of the studies considered participants with different language backgrounds, involving at least three spoken and/or sign languages. Examples include the following across multiple participants: five sign languages (Kusters et al. 2021), 13 (Cannon et al. 2020), eight (Guiberson 2014) and seven (Teschendorf et al. 2010; Waltzman et al. 2003). However, few studies went beyond mere lists by reporting more in-depth and nuanced data (e.g., English and Spanish language support at home and at school across their study participants; Bunta and Douglas 2013). As well, because the majority of studies (25 out of 33) were from English-dominant countries, English spoken language and ASL and Auslan were overrepresented in the sample. Twelve studies did not report the languages utilized by the DML population of interest at all.

Even fewer studies reported the country of origin of their participants; those that did often included participants from a large number of different regions (e.g., Alfano 2019, with learners from a number of Central American countries). The country where the studies took place, in contrast, was consistently reported and/or evident from the background of the researchers and publication venue (Table 4):

The majority of studies (16) took place in the United States, while four each were conducted in Canada and Germany. Three stemmed from Australia, while the remaining studies were the only ones in the current review in their respective countries: England, Finland, Kenya, Norway, Scotland, and Spain.

5.4 Research participants involved

Since the population of interest was not always the same as the research participants providing information (Table 5), these categories are considered separately. The sample size of participants was quite varied, ranging from one (three studies) to 65, (average = 16.1; median = 9; SD = 17.3). Since studies in medical clinics tended to have larger sample sizes, ranging from 12 to 56 participants (average = 25.3; median = 16.5; SD = 20.6), a more representative portrayal of participant numbers would exclude clinical studies. For the purpose of this review, we do not include numbers of monolingual d/Dhh participants in comparison groups. In the 29 non-clinical studies in this review (average = 14.8; median = 7; SD = 16.9), a full 18 studies (i.e., over half) involved ten or fewer participants, meaning that all research was informed by a total of 432 participants. Further, if only data gathered from DML themselves is considered, the number sinks to 118 participants in non-clinical and 101 in clinical studies.

Regarding research participants, many researchers consider data gathered from the learners themselves (19 studies), whether this be through testing, observation, interviews, a review of school documents, or a combination thereof. Some studies also involved parents or the whole immediate family of the d/Dhh learners, teachers, principals, interpreters, further service providers, or peers. Of the 33 studies involved, 21 examined only one group of research participants; however, eight studies gathered data from two different populations, while two studies each looked at three or four different participant groups.

5.5 Sampling procedures

Closely tied to research participants is the question of data sampling, i.e., how the participants were chosen for the study (Acharya et al. 2013). This important information was not addressed directly in all studies. In total, 28 articles explained sampling procedures. Five studies gave no information about sampling procedures.

All studies reported involved non-random sampling (i.e., data was gathered based on the researcher’s sample construction possibilities), with participants often self-selecting. For the purpose of this scoping review, we categorized studies into one of four sampling approaches: (a) convenience sampling (i.e., researcher has access to the participants), (b) purposive sampling (i.e., an a priori decision on necessary traits that participants must have, and recruitment of those who fit the category), (c) snowball sampling (i.e., participants are asked to recruit further participants), and (d) quota sampling (i.e., participants are chosen to represent specific sub-groups of the population) (Table 6):

Fifteen of the 28 studies used convenience and six studies used purposive sampling, while sampling procedures were combined in six studies. Pre-existing contacts with schools, DML’s teachers or families were drawn upon; in many cases, these were private contacts in family or friendship circles, and in one case, the researchers themselves were part of the group under study (Kusters et al. 2021). For studies combining sampling methods, a two-step procedure of gathering a group of participants was common. This usually involved a general invitation to participate (e.g., parent-teacher evenings, notice boards, mailing lists, teacher conferences, social media), combined with convenience and/or purposive sampling. One study carried out quota sampling as a primary sampling method (Begon 2021). Only two studies specifically mentioned snowball sampling, which was combined with purposive (Shin 2003) or purposive and convenience sampling (Batamula 2016).

5.6 Research design

To describe the research design, we considered various aspects, although the first two were not always stated clearly in the project. We looked at (a) the theoretical paradigm underlying the study motivation and research design; (b) research paradigm as being deductive-nomological (i.e., hypothesis-testing) or explorative-interpretative (i.e., hypothesis developing); (c) whether the study involved an experimental design; (d) type of data collected (i.e., qualitative or quantitative); (e) method of data collection; and (f) instruments involved (Table 7; for a full listing, see Table 7a in Supplementary Material):

Because the four studies carried out in medical clinics seemed to form a category of their own, they are not included in the following results. These studies were all nomological, non-intervention studies that elicited quantitative data based on a battery of standardized and normed tests to consider different aspects of DML’s hearing and language levels in the language of the wider community. All but one (Waltzman et al. 2003) considered a comparison group of d/Dhh learners who were not multilinguals as per the definition in this review. The studies are included in the Tables in the Supplementary Material.

First, it was challenging to determine the theoretical approach that researchers took when designing their studies. A few studies placed themselves in ethnographic research, some mentioned a phenomenological approach or a design based on grounded theory, others an intersectional methodological approach; however, in almost all cases, we could only determine the approach by inferring the researchers’ standpoints. Thus, we decided to dispose of this category for the analyses of research paradigms as being too unclear to delineate satisfactorily.

Second, the research paradigm invoked was tied to the field of linguistics involved and the research questions posed. Most evident, for example, was that clinical studies were solely quantitative in nature and involved batteries of tests. We classified studies according to whether they – explicitly or implicitly – were primarily hypothesis-testing (deductive) or had the goal of hypothesis development (explorative). A full 26 of the non-clinical studies were classified by the researchers or through an analysis of the research design as explorative, while only three took a nomological perspective. Many of the projects analyzing quantitative data also invoked explorative research paradigms, although projects focusing only on quantitative results tended to pose hypotheses in some form. All projects considering only qualitative data were explorative.

Third, only four studies conducted intervention research, of which three were single case design studies carried out by the same research teams. These involved small numbers of participants with between one and five learners, as to be expected in such heterogeneous populations. Finally, the primary data gathered and analyzed through the study was generally qualitative, with 21 studies gathering only qualitative data and a further four studies both qualitative and quantitative data. The remaining five (non-clinical) studies focused on quantitative data and tended to have slightly larger participant numbers. In almost all cases of quantitative data analysis, the researchers chose to present only descriptive data (i.e., generally raw data and averages), but eschew inferential statistical testing – unsurprising considering the small number of participants involved and thus the difficulty in achieving sufficient power to yield statistically significant results.

To describe the data collection chosen for the studies, we described studies along two basic categories: (a) data gathering technique (observational, survey, educational records, or testing), and (b) type of instrument used (i.e., videography, field notes, interview, questionnaire, school records; Table 8):

Twelve projects used only survey data, three only observation data, and eight assessment data (including four clinical studies), whilst ten projects combined survey data with one or more other methods of data collection. Of these, one project combined interviews with videography, four with field observations, two with the examination of school records, and one single-case study that used data from questionnaires, interviews, school records, videography, and assessment (Granda 2014).

Of the survey methods used, five projects invoked questionnaires as collection instruments, whilst 21 projects used (mostly semi-structured) interview techniques, and four projects combined interviews with questionnaires. For the observational data, only two projects included videography as a method of data collection (de Garcia 1993; Granda 2014), and nine used participant observation with field notes. Three studies included school records, often retrospectively, and eleven studies in total involved testing of one or more of the languages used by the learners.

Central to the data collection method and instruments is the type of information to be gleaned from the data. For projects using surveys, for example, at least one of three aspects was at the forefront. Studies using interviews with teachers and other service providers as one of the chosen instruments were generally interested in educational perspectives and educational needs of DML. Projects where the learners themselves or their parents were interviewed often had a focus on DMLs’ communication and language use. Finally, data collected through interviews with parents or service providers focused on family decisions about language use, medical interventions, and/or educational choices, such as which schools the children should attend. Observational data, in contrast, was collected solely to explore learners’ communication skills and language use in context within the home, school, and community settings.

Some studies utilized standardized instruments keyed to specific languages to examine various aspects of language comprehension and production and, in one case, auditory measures, or ad hoc instruments developed by the researchers to examine one aspect of language, such as grammar, reading comprehension, or vocabulary. Only five out of 33 studies assessed the language proficiency of the participants in the study with standardized data collection assessments (Table 9 in Supplementary Material). Nine other studies examined an aspect of language (e.g., vocabulary knowledge, receptive auditory skills) but did not provide assessment results indicating overall language proficiency. And five out of those nine studies reported language proficiency via parent report or interview, but did not assess the learners themselves. Four studies reported on the language proficiency of DMLs in their second language, with one reporting information from parent interviews about both the second (Korean) and third (ASL) languages utilized by the learners (Shin 2003).

As with the previous categories of interest, data collection methods were dependent upon both the number of participants involved (the fewer the number, the more instruments), and the primary question of interest in the study. A combination of instruments for in-depth qualitative data, but with a focus on interview data, seemed to be preferred, while methods such as videography or examination of school records were much less common.

6.1 Research questions

The research questions posed across a third of the studies reviewed ask for more information about the DML and their families. This aligns with recommended methodology when investigating a new phenomenon or population. Understanding the characteristics of DML can assist professionals in providing linguistic supports that might increase language and literacy skills, as well as social-emotional development. The majority of studies asked research questions about the family’s, educational professional’s, and learner’s perspectives about the educational system and their decision-making processes. These research questions can assist professionals in ensuring they are using family-centered practices that engage them in the DML’s language acquisition and schooling.

Understanding the backgrounds of DML and the perspectives of professionals provides the field with a foundation for culturally and linguistically diverse strategies and practices. Yet, only eight studies contained research questions about language and literacy interventions with DML. This highlights the minimal research on linguistically and culturally responsive teaching practices with DML.

6.2 Population under consideration

In the investigation of low-incidence, heterogeneous learner groups, accurate descriptions of the population of interest are important. We reviewed the population under consideration and found that only half of the studies actually focused on learners, while the remainder considered the family and educators working with these learners. For example, only just over half of those studies reported the age at which the learner and/or their family immigrated to their current country, even though this is a vital piece of information to understand the cultural and linguistic background of the DML and their family. Results showed that many studies did not collect enough data to offer a holistic picture of DML, including more detailed information about their length of time and exposure to schooling and language.

A holistic perspective of DMLs’ backgrounds can then provide perspective on how the multiple languages of participants may impact the methodology of the research and allow researchers to consider how they use methodology to mitigate these variables and strengthen their research design. Understanding the language and the culture of the population being considered can help researchers realize how they may impact the way people respond to surveys, interviews, interventions, etc.

A gap within the studies reviewed includes asking questions and reporting on whether or not DML had a disability. Researchers must consider that 40–50% of d/Dhh learners have disabilities (Gallaudet Research Institute 2013), which may have an impact on language acquisition, as well as the variations in services and supports across countries. This again highlights the importance of providing as much information as possible about all DML research participants.

6.3 Languages and cultures involved

The large disparity between the number of spoken languages and sign languages may indicate that heritage sign language is not accessible in some countries (i.e., no formalized sign language and/or courses for instruction), professionals are not encouraging use of them by families, and/or families have chosen listening and spoken language for their child’s mode of communication and do not expose them to their native sign language.

Of interest is that all studies reported the use of at least one spoken language, but only 18 studies also reported DML participants using a sign language; and of these, 14 report only the sign language of the host country community. The languages considered were almost solely English or ASL, the two dominant languages in the United States and Canada, where the majority of the work was published. Considering that across all 33 studies, a total of 44 different spoken and sign languages were utilized by the participants, we seem to know very little about the linguistic profiles of DML involved.

6.4 Research participants involved

We considered research participants regarding both number and type. In general, the number of participants was quite small, ranging from single-case studies to a maximum of 65 individuals. However, over half of the studies involved under ten participants. Since many participants were not DML themselves, our knowledge of the entire population is informed by direct data from only 219 DML participants – 101 of whom were involved in clinical studies – in 19 studies. Logically, the number of participants was commensurate with the methodological design of the study (e.g., three single-case design studies with under five participants; case studies with one participant).

Regarding participant groups, most studies looked to only one population, whether DMLs, parents, teachers, or other service providers, although twelve studies did turn to two or more groups – often, but not always, involving a combination of learners and their parents and/or families. Very few studies gathered data from more populations, such as learners, parents, and teachers, and only five studies compared DML with a comparison group of d/Dhh learners whose families did not speak a heritage language at home. The choice of research participants was also related to the research design, with testing involving only the learners, interviews being the mainstay for data from parents or educators and further service providers such as school principals or speech-language pathologists, and observations for data from families.

The small numbers of research participants – especially of DML themselves – and the tendency to focus on only one population reflect the difficulties involved in studying low-incidence populations. However, this narrow focus raises questions about the validity of the information available to date. This becomes especially relevant when considering issues of sampling.

6.5 Sampling methods

The studies considered here relied completely on non-random sampling and, specifically, most often self-selection of participants in convenience sampling procedures. This is unsurprising considering the nature of the diverse, low-incidence population of interest and the choice of qualitative research designs in the majority of projects. As well, it reflects the tendency of most educational research on specific target groups. While no sampling procedure is “better” or “worse” than the others, but rather depends on the accessibility and diversity of the population being studied and the research questions of interest, it is important that a research field is informed by data gathered through a variety of sampling procedures in order to ensure that a wide variety of situations are considered. This is not the case with the group of studies on DML. Thus, results as a whole must be interpreted with caution.

6.6 Research design

With the exception of the clinical studies, which carried out group comparisons on quantitative data, almost all studies were explorative and non-experimental (with the exception of three single case design studies) which elicited qualitative data. The decision to conduct exploratory, qualitative research is most certainly tied to the dearth of information in general on DML. However, it does point to some obvious research gaps, which it would be advisable to address in future research.

Notably, there is a major reliance on introspective data, which is useful for gathering information on attitudes and motivation, but less valid when considering aspects such as linguistic behavior or proficiency levels, where methods such as observation and assessment are preferable. The lack of observational data is most likely due to its strain on both personell and financial resources, requiring a large number of researcher hours and, in the case of video data, expensive technical support. Such issues are not as relevant when using assessment instruments, but these come with their own caveats. Most prominent is the fact that there is a general lack of research instruments available for the DML population. This resulted in the use or adaptation of instruments – both survey and assessment – developed for other populations and lacking norms for DML. Confounding this difficulty is that most instruments were used without consideration for difficulties arising through the fact that test-takers were learners of the test language, meaning that difficulties in the tests may actually be due to difficulties understanding test items and instructions. Of particular concern is that, although all studies were concerned with language experiences of DML, very few even noted their language abilities or reported any language assessment data, and of these, sign language assessments were extremely rare.

Finally, even survey data is not without its problems. In a population as heterogeneous as DML, survey data may not be able to portray accurately the complexity of learners’ situations, since respondents are typically answering questions in their second or third language and may not provide all information if they do not trust/know the researchers. This should also be considered when evaluating information stemming from such studies.