Investigating the relationship between the speed of automatization and linguistic abilities: data collection during the COVID-19 pandemic

2.1.1 Recruitment

Recruiting adult participants for online studies is possible through platforms such as Amazon Mechanical Turk (https://www.mturk.com) and Prolific (https://www.prolific.co). However, recruiting children and special populations is more difficult, even in normal circumstances. There are platforms that connect families to available research studies, for example, Lookit (Scott and Schulz 2017) and Children Helping Science (https://childrenhelpingscience.com), but in this study, we found that sharing our study with parents and guardians on social media provided the most success in recruiting participants.

We recruited participants through asking schools to share information with parents and guardians, and we advertised through social media platforms such as Facebook and Twitter (now X) providing a wider geographical reach. This saved time and travel expenses and was particularly advantageous in recruiting children with DLD. Whilst DLD is a common disorder, affecting approximately 7 percent of the population (Norbury et al. 2017; Tomblin et al. 1997), unfortunately it is not a very well-known condition. Therefore, it can be difficult to find children with DLD who are willing to take part in scientific research, especially as the very tasks being assessed are those that children with DLD find difficult. Facebook offered the opportunity of joining specialist groups such as “Keeping children entertained during lockdown”, and DLD parental support groups, both of which generated positive interest.

2.1.2 Adapting tasks to an online format

We used a hybrid approach to the experimental tasks where the researcher communicated with the child using the videoconferencing platform Zoom (https://zoom.us). This interaction was important as it allowed the experimenter to explain each task and assess understanding as they would have done in person. It also enabled the experimenter to monitor progress and encourage and motivate children through each of the online sessions.

Many of the tasks in our study relied on standardized cognitive and language assessments which are usually administered in person. Permission was sought from the publishers of the assessments, and they recommended using a digital visualizer to project tasks to participants, using the screen share option on Zoom. Further information as to how each of the tasks was adapted for online use is provided in Sections 2.3.1 and 2.3.2.

2.3.1 Cognitive tasks

Multiple-trial Tower of Hanoi (MToH) task: Both the experimenter and the child had a physical puzzle in front of them at the start of the task. The experimenter explained the rules and showed the child examples of legal and illegal moves. Following Beaunieux et al. (2006), the experimenter showed the child the first move by putting the smallest disc on the middle rod of the puzzle. Children were encouraged to ask questions to verify their understanding and were invited to do a practice trial before starting, to learn the rules of the task and to get used to manipulating the discs. Participants were asked to solve the puzzle 30 times, with a secondary task in the last five trials. The secondary task involved tapping in response to a recording consisting of a random sequence of the words one and two, with a four-second pause between the words. Participants were asked to solve the puzzle with their dominant hand and then tap with their non-dominant hand in response to the numbers (two taps in response to the word one and one tap in response to the word two). The aim of the secondary task was to test automatization: if the procedure for solving the puzzle is fully automatized, then doing the secondary task should not interfere with performance on the puzzle. For every trial, we calculated the amount of time to complete the task and the number of moves used. As children repeat the task, their time per trial starts to reduce and therefore, after several trials, the experimenter would tell them their time. This was motivating for participants as they enjoyed trying to improve their time and parents and guardians often shared in this success. Videos 1 and 2 demonstrate the different stages of the MToH task: the cognitive phase of the task (associated video-1-blake & dabrowska.mp4) and the autonomous phase of the task (associated video-2-blake & dabrowska.mp4).

Raven’s Coloured Progressive Matrices (Raven et al. 1938): This is a non-verbal intelligence test for children from age 5 to age 11. It consists of 36 items in three sets of 12 problems to assess the development of intellectual maturity. Each diagrammatical problem has a “piece” missing and the participant is required to select the missing piece from one of six options. The child selects their answer by naming the number corresponding to the missing piece of the puzzle.

Backwards colour span task (Riches 2012; Zoelch et al. 2005): This is a working memory measure involving a visual variation on the backward digit span task. The child watches as various coloured tennis balls are placed in a physical tube (see Figure 3). Their task is to work out what order the balls will come out of the tube. Nine colours are used which correspond to digit recall stimuli. These are: black, white, red, blue, green, pink, grey, and brown, as equivalent to eight one-syllable stimuli, and one two-syllable stimulus: yellow, corresponding to the number seven. The task comprises blocks of six stimuli: block 1 with two items per stimulus; three items per stimulus in block 2; four items per stimulus in block 3; five items per stimulus in block 4; and six items per stimulus in block 5. To pass a block, the child must achieve four out of six correct responses, and after three failures in a block, the test is discontinued. Before starting, the experimenter showed the child all the colours and ensured that they were able to name each colour before proceeding. This task was adapted for online use by using a digital visualizer and screen sharing to present the physical tube and coloured balls. Real objects were used to try and make the task more enjoyable for the child.

Figure 3:

Backwards colour span task.

Embedded triplet task (adapted from Arciuli and Simpson 2011): This task was developed for our study on Gorilla Experiment Builder (https://gorilla.sc; Anwyl-Irvine et al. 2020). Gorilla is a cloud-based platform enabling researchers to create and run online experiments through a graphical user interface which means that programming knowledge is not required. Furthermore, additional support is available from Gorilla’s scripting consultancy service. Participants access tasks on the Gorilla platform through a link which launches the task without the need for the participant to download additional software. The embedded triplet task used in our study is an implicit statistical learning task which involves a familiarization stage and a test stage. During familiarization, participants see cartoon figures of 12 friendly aliens which appear on the screen one by one. The aliens form triplets which are always presented in the same order; the order of the triplets is random. In the test phase, four new “impossible triplets” were created by combining one alien from each of the original triplets. In each trial, the child is presented with one of the original triplets and an impossible triplet, that is, six stimuli in total, presented sequentially with a short pause after the first triplet. The child’s task is to choose which triplet had appeared previously during familiarization. There are 64 test trials, and the entire task takes about 15 min. In the original task designed by Arciuli and Simpson (2011), participants responded verbally during the test phase and their response was recorded manually. In our online task, participants made their choice by clicking on one of two options (A or B), and their responses were scored on the Gorilla platform.

2.3.2 Language tasks

The experimenter used a digital visualizer to project language tasks to participants, using the screen sharing option on Zoom. The use of a visualizer allowed us to present printed materials to the child such as standardized language tasks which often involve a picture selection format. Where a child would normally point to a picture to indicate their choice, we asked them to say the number of the picture that matched the sentence or word (as applicable). These are minimal variations imposed by the online format. As many of the measures in our study are standardized tasks, we had to ensure that the testing was as similar as possible to face-to-face testing. In each instance, the task was explained to children, and they were able to do a few practice trials before beginning the task. The language assessments used are described below.

Test for Reception of Grammar (TROG-2; Bishop 2003): TROG-2 is a receptive language test which evaluates understanding of grammatical contrasts which are marked by inflections, function words, and word order. The format comprises a picture selection task consisting of 80 four-choice items. Grammatical contrasts are grouped in blocks of four items and a block is passed if all four items are answered correctly. The experimenter reads a sentence aloud to the child and they are asked to select the correct picture in response to the sentence.

British Picture Vocabulary Scale (BPVS3; Dunn et al. 2009): The BPVS3 assesses a child’s receptive vocabulary through a picture selection task format. Each item in the test consists of four illustrations and the child’s task is to select a picture that best illustrates the meaning of a stimulus word spoken by the experimenter. There are 14 sets of 12 test items, which increase in difficulty.

Recalling sentences sub-test of Clinical Evaluation of Language Fundamentals (CELF-5; Wiig et al. 2013): This task evaluates the ability to listen to and repeat spoken sentences which increase in length and complexity. The experimenter reads the sentences aloud and asks the child to repeat the sentences without changing the content. The test is discontinued after five consecutive sentences with four or more errors in each sentence.

Narrative task: The Frog Story (based on Frog, Where Are You?; Mayer 1969): The narrative task involves the experimenter showing the child The Frog Story, a short picture story without words. The original story contains black and white pictures, but we used an adaption with colour illustrations by Monica Pascual Holder (made available by Esther Pascual). The child is advised that they will see the picture story twice, and on the second occasion, the child is asked to tell the story in their own words. The narrative is recorded for transcription purposes.

Expository discourse (Nippold et al. 2008): In this task, the child explains their favourite game, sport, or hobby. The experimenter uses specific prompts to encourage discussion; for example, the experimenter asks the child to pretend that they are talking to an alien from outer space, who has no knowledge of how things work on planet earth. The child’s response is recorded for transcription purposes.