Development and psychometric properties of the adolescent risk behavior questionnaire

Introduction

During adolescence, various causes of disability and mortality are related to behaviors that are harmful to health or accidents, which are preventable causes [1], 2]. Risky behaviors are characterized by being associated with an increase in harmful consequences to health. This is relevant in adolescence, which is characterized by the search for sensations, novelty, experimentation, development challenges, reward, among other features [3]. For most adolescents, this results in growth and development; however, exploratory behaviors can be dangerous [3].

There are various investigations that approach the study of risky behaviors from different perspectives. For example, it has been proposed that their incidence follows an inverted curve pattern, beginning in early adolescence with a tendency to increase, and decreasing in late adolescence [4]. Likewise, it has been observed that public health practices [5] and social norms influence risky behaviors in adolescents [4]. Another perspective is their co-occurrence pattern, where one risky behavior is more likely to be accompanied by other risky behaviors [6]. Neurobiologically, the dual model of adolescent brain development has been proposed, which states that the maturation of the prefrontal cortex, which provides executive functions, consequence evaluation, self-control, etc., develops later, and therefore shows a lag with the development of other brain areas, especially the limbic system, responsible for sensation seeking and emotionality; resulting in a propensity for impulsive behaviors or involvement in risky behaviors [7], 8]. It has been observed that the reward system is more sensitive during adolescence than at other stages of development [4]. Likewise, it has been observed that the hormonal influence of puberty is closely linked to risky behavior and aggression [8].

The main risk behaviors during adolescence correspond to risky sexual behaviors, consumption of illicit and legal substances, traffic accidents, teenage pregnancy and academic dropout [1]. Morbidity and mortality rates increase between 200 and 300 % from childhood to adolescence in association with risk behaviors, taking for example 70 % of annual deaths of adolescents in the United States derived from risk behaviors, such as reckless driving [9]. Mexico is in first place for teenage pregnancy among women aged 15 to 19 within the countries of the Organization for Economic Cooperation and Development (OECD) with 13.7 births per 1,000 women [10]. ENCODAT 2016 reports [11], 12], focused on the population aged 12 to 17, indicate that 6.5 % tried electronic cigarettes, 39.8 % consumed alcohol at some point in their lives, 15.2 % had an excessive alcohol consumption in the last year, 5.3 % tried marijuana and 6.4 % any illegal drug. In 2020, deaths from motorcyclist accidents increased 28.6 % within the 10 to 19-year-old group [13]. In 2015, 7,785 adolescents were admitted to state treatment or detention centers for antisocial behavior associated with common law crimes [14].

There are various instruments for measuring risk behaviors [1]. The Adolescent Risk Behavior Screen (ARBS) [15], consists of nine items, developed from a large-scale study involving a sample of 16,664 adolescents aged 14–18 years. Various items in this instrument identify attitudes and characteristics associated with risky behaviors, without focusing solely on whether the risky behavior has been carried out or not. The Adolescent Risk Inventory (ARI) [6] is another instrument that was developed in a clinical population with psychiatric pathology, in which 134 adolescents participated, composed of 29 items, focusing mostly on suicidal risk and attitudes towards the sexually transmitted disease HIV; leaving aside aspects of antisocial behavior, reckless driving or substance use. As for the Substance use and abuse subscale of the Problem Oriented Screening Instrument for Teenagers (POSIT_SUA) [16]; it is a subscale composed of 17 items with dichotomous scoring (true/false); this comes from the original instrument composed of 139 items, with dichotomous responses, includes 10 subscales, and was developed in a sample of 1,000 adolescents aged 12 to 19, as part of a long-term research project of the National Institute on Drug Abuse [17]. Another instrument, the Youth Risk Behavior Survey (YRBS) [18]; is used to collect reliable public health data related to risky behaviors in adolescents; it is part of the Youth Risk Behavior Surveillance System; it is composed of approximately 92 items, its objective is to screen for behavioral problems that could lead to unintentional injuries or violence, risky sexual behaviors, drug, alcohol or tobacco use, physical inactivity and unhealthy dietary patterns.

Despite the above none have been validated in the Mexican population [3], or they are very extensive [18], which may make them less likely to be used in clinical settings and non-clinical settings [19], some instruments do not measure very important risk behaviors, or others focus on addictions [6] or suicidal behavior, despite there being specific screening instruments for these health problems [20], which is why the main objective of this research was to develop a free, brief and simple instrument that measures the main risk behaviors, and test its psychometric properties.

Materials and methods

The first stage of this research focused on developing the instrument, the main purpose of which was to screen whether the adolescent has engaged in the main risk behaviors. This first stage was initiated by two of the authors of this research, child psychiatrists with experience in the clinical management of inpatients and outpatients. A literature review of available measurement instruments was conducted, in addition to an in-depth analysis of the main risk behaviors in adolescents. The initial intention was for each item to be short, with easy-to-select responses and wording geared toward the adolescent population. This was also in line with the recommendation that an instrument of this nature should have no more than 20 items [19].

Risk behaviors were our latent variable, and the dimensions were created based on prevalence and/or related categories; the degree to which the adolescent has engaged in the risk behavior was also considered as an indicator [19]. The responses were written in the style of the Child Depression Inventory [21], which are phrases that implicitly contain the ordinal measurement level, with values ​​of 0 (equivalent to: conduct not performed), 1 (equivalent to: mild behavior performed) or 2 (equivalent to: conduct performed). It is important to clarify that the item developers were not part of the group of expert judges for content validation.

This initial stage resulted in a first preliminary 16-item instrument, which was examined for content validity by a group of five experts. The selection criteria were: being independent of the instrument developers, having clinical experience in the field of adolescent mental health, and additionally having postgraduate academic degrees and/or experience in postgraduate teaching. Based on the above, the group was made up of one psychiatrist with a doctorate in neuroscience, one psychologist with a master’s degree in systemic family therapy, two child and adolescent psychiatrists with experience teaching in medical residencies and one neonatal pediatrician, director of teaching and medical education, with multiple research products.

To determine content validity, each judge rated each item on four characteristics: Clarity, Pertinence, Relevance, and Sufficiency; using Likert-type ratings on four ordinal levels (poor, fair, good, excellent); intentionally leaving four rating options to avoid neutral ratings, in accordance with recommendations in the literature [22]. The comments and observations provided by the judges were considered, and subsequently, the ratings were integrated to determine the Aiken V content validity index, resulting in the final preliminary instrument.

In the second stage of the study, A cross-sectional study was carried out, with non-probabilistic convenience sampling, obtaining a sample of 100 adolescents. Regarding the calculation of the sample size, there’s no consensus regarding the required sample size for psychometric studies; however, recommendations are available, but they vary widely. A systematic review reported that 10 % of studies determine the sample size a priori [23]. Generally, the recommended sample size is subject/item, ranging from 2 to 20. The minimum sample size was 100–250 participants [23], 24]. Factors that influence sample size include distribution, robustness of data [23], and population type; very few studies are conducted in clinical populations, where accessibility may be difficult, resulting in a smaller sample size [25]. The selection criteria were: adolescent participants aged 10–17 years, regardless of sex, whose native language is Spanish and attended an outpatient clinic at the Pediatric Hospital; who did not have any medical condition that would prevent them from reading, understanding, or responding to the scale; and who also gave their assent, along with the agreement of their parents or guardians, by signing an informed consent form. Descriptive statistics were carried out mainly with frequencies and percentages. For the psychometric analysis, internal consistency was calculated using Kuder Richardson 20 and Cronbach’s alpha. Content validity was carried out through a literature review, opinion of five experts and Aiken V index. Exploratory factor analysis was initially carried out with the calculation of sample adequacy indices such as KMO and Bartlett’s Sphericity Test, subsequently the factors were extracted using the principal components analysis technique and varimax orthogonal rotation. Confirmatory factor analysis obtained the distribution of the sample, identification of outliers, factor loadings, error ratios and goodness-of-fit indices. Excel, SPSS 24 and AMOS were used.

This research was approved by the Local Health Research Committee 1,302, granting the institutional registration number R-2022-1,302-049, September 5th, 2022.

Results

The age range of the 100 participants was 12–17 years, mean age 14.4 (1.3), 72 % were women. 95 % presented a mild risk behavior, 53 % presented a serious risk behavior and 87 % presented at least two risk behaviors; by sex64 % of men presented serious risk behavior. Of the total sample, 69 % had already consumed alcohol, 21 % some illicit drugs, 14 % reckless driving, 77 % poor quality of sleep, 37 % playing hooky, 25 % related to dissocial peers, 11 % reckless motorcycle driving and 7 % risky sexual behavior.

On the first stage, which was the development of the scale by two of the authors, a 16 item first preliminary instrument was obtained, then, it was reviewed and rated by the group of experts. The comments and observations provided by the experts were taken into account, which resulted in minor wording modifications to five items. Once the qualitative suggestions of the judges were integrated, a quantitative analysis was carried out with the Aiken V statistic, to obtain the coefficient of content validity, which was 0.96 (Figure 1).

Figure 1:

Final version of the 12-item risk behavior questionnaire in adolescence (CCRAC)

The reliability obtained through the internal consistency of Cronbach’s alpha for the instrument with 16 items was 0.73; however, due to poor values ​​in the exploratory factor analysis, four items [11], [12], [13, 15] were eliminated.

The reasons why the items were removed was: item 11 (Regarding cell phone and computer use, without taking into account the time you spend on homework or listening to music: Do you use them more than 2 h a day?) obtained a very low load of 0.23, and furthermore, it loaded in component 1 (high prevalence), and not in component 4 (wellness). After item 11 was removed, component 4 (wellness) showed inconsistencies, with three items of other components sharing borderline low factor loadings with component 4 (1 Alcohol, 10 Couple and five playing hooky; 0.34, −0.30 and 0.36 respectively); likewise, items 13 (diet) and 12 (extracurricular) loaded simultaneously on their theoretical component (wellness) but also on component 3 (accidents). In addition, item 15 (Do you have any goals, dreams, plans, or purposes in life?) was the only item with unique loading in component 4 (0.711). Therefore, item 15 was eliminated; resulting in item 12 having a unitary load on component 4, it was removed; and the same procedure was followed with item 13 (Do you have a healthy and balanced diet?).

The final version of 12 items obtained a Cronbach’s alpha coefficient value of 0.79; and McDonald’s ω 0.80. The reliability obtained through Kuder Richardson 20 for dichotomous screening purposes (present/absent), taking into account the Likert rating 1 or 2 as present, was 0.72, and taking into account only the Likert rating 2, it was 0.76.

The exploratory factor analysis of the final version of 12 items was performed using the principal components method and varimax rotation; the indices obtained were KMO 0.76, Barlett’s sphericity test (X² (66) = 361.3, p<0.001), indicating the feasibility of the analysis. Three components were obtained, with loadings ranging from 0.50 to 0.80, and a total explained variance of 56.9 %. These components are consistent with those reported in the literature, naming the first component High prevalence, as it contains items 1 Alcohol, 2 Tobacco, 3 Drugs, 4 Sex and 10 Partner, the second component Dissocial containing items 8 Dissocial peers, 9 Dissocial behaviors, 5 Playing Hooky and 14 Reduced sleep; while the 3rd component Accidents contains items 6 Car driving, 7 Motorcycle driving and 16 Adult supervision.

A confirmatory factor analysis was carried out, eliminating two participants due to atypical scores based on the Mahalanobis distance [14]; After that, it was determined that the sample did not present a normal distribution, with Kurtosis values ​​of 36.8 and critical ratio 9.9. Therefore, a bootstrap resampling technique was used [26], computing 1,000. Obtaining an average X² 59.6, and the average obtained in the present model was X² 61.3; with a Bollen-Stine Bootstrap statistic p=0.423; demonstrating that the proposed model is correct, considering its non-parametric distribution. The proposed three factor model obtained five goodness-of-fit indices with favorable values ​​χ²/gl 1.3, χ² (61.3, gl 47 p 0.078), TLI 0.93, CFI 0.95, RMSEA 0.56 (Figure 2).

Figure 2:

3-Factor model of the adolescent risk behavior questionnaire

Discussion

Risk behaviors are highly frequent in the adolescent population, which leads to adverse health outcomes. Our study indicates that the vast majority of adolescents have presented risk behaviors [9], coinciding with various morbidity reports [27], prevalence of consumption [28], the age of onset of alcohol and drug consumption is usually adolescence [29] with various studies; however, differing with what was mentioned by Willoughby and collaborators, who report that although there is involvement in risk behaviors in adolescence, they are not as prevalent as believed [2].

The reliability of the instrument is good, whether used as a questionnaire or a scale, according to the reference values ​​of 0.7–0.9 [19], 30].

The final version of the CCRAC with 12 items showed good content validity, according to values ​​proposed by Aiken [31]. It also demonstrated adequate construct validity, with KMO indices and Bartlett’s Test of Sphericity indicating that it was an adequate sample for factor extraction [25], 32], with items whose loadings were>0.30 [33]; obtaining three factors that together exceed the minimum percentage required for explained variance [24]. The proposed model could be confirmed, since the three factors adequately adjusted to the data obtained, reflected by at least five indices obtained (χ2, χ2/df, CFI, TLI, RMSEA), whose values ​​comply with those proposed by various authors [34], [35], [36], [37].

The factors obtained coincide with high-risk behaviors that tend to occur simultaneously and lead to favorable health outcomes [1]. Studies indicate that substance use begins at an early age, and the gateway phenomenon is observed, where the initiation of a substance increases the risk for the use of another substance [38], [39], [40]; and that together with risky sexual behavior, which in Mexico is a public health problem with a rate of 70.6 per 1,000 pregnancies, compared to the OECD average of 13.7 [10], they are highly prevalent, making sense that they are grouped into a high prevalence factor. Along the same lines, there are multiple reports that indicate a prevalence of up to 71.4 % of violence and abuse in dating relationships. Regarding the high impact factor, it is the one associated with vehicular risk behavior whose grouped items have been shown to be related to each other, in accordance with what is reported by various authors [40], 41]according to the WHO road safety report [42]. The negative social impact agrees with the theoretical support that has been reported in longitudinal research [43], [44], [45].

Regarding the Accidents factor, it is associated with risky vehicular behavior, whose grouped items have shown to be related to each other; which is in line with what was reviewed by Lopez-Araujo (2007) [42]; who reports that reckless driving, high speed, alcohol consumption and taking risks are very frequent characteristics in young people who have had accidents [41], 42], and for its part, the road safety report from the WHO [43] emphasizes that young people are the most frequent in presenting carelessness and risks in vehicular driving.

The third factor obtained, Dissocial, agrees with the theoretical support reported in longitudinal research; since Farrington (2015) has pointed out that continuing to interact with dissocial peers is a risk factor for presenting criminal behavior; the same has been pointed out by Zara (2009) [44], [45], [46]. In a cohort study of 2,767 adolescents, a bidirectional relationship was found between sleep problems and impulsivity, which makes them more prone to risky behaviors and present self-regulation difficulties [47].

Regarding the items that were eliminated, 11 excessive cell phone use, 12 participation in extracurricular activities, 13 healthy eating and 15 having some goal in life; although they had adequate qualitative properties of content validity (clarity, pertinence, relevance and sufficiency); psychometrically they showed very poor values; this may be due to various issues [24]; in the case of item 11, being very prevalent in all conditions, it contributed factorial load in all components; and in relation to the remaining ones, it was because a single item is not enough, and therefore more items focused on measuring the underlying constructs would be necessary.

The CCRAC instrument has qualitative similarities to the Adolescent Risk Behavior Screen (ARBS) [15]; at the beginning of its development it consisted of 99 items and in its final version were nine items, with Likert-type ratings. It has content and criterion validity, however; no relevant psychometric properties were reported in terms of reliability and construct validity. In addition, a substantial difference lies in the fact that five items focus on substance use and two items focus on psychopathological symptoms; which, although they are related phenomena, are different constructs.

Likewise, the goodness-of-fit indices obtained with the CCRAC are similar to those obtained by the Adolescent Risk Inventory (ARI), which were X²/df=1, TLI=0.984, CFI=0.991, and RMSEA=0.021 [6]. However, there are differences with ARI, on the one hand, most of its items are only dichotomous, and two subscales are composed of two items, and includes items focused on suicidal behavior and symptoms of eating disorders, for which there are screening instruments more targeted to these symptoms such as ASQ [20], Okasha Scale [48] or the SCOFF questionnaire [49].

CCRAC partially coincides with the psychometric values ​​of the substance use and abuse subscale of the Problem Oriented Screening Instrument for Teenagers (POSIT_SUA), which in a validation study [16] in 569 Spanish adolescent students aged 12–18 years; obtained an internal consistency through KR20 0.82, while CCRAC 0.72 and 0.76, being an acceptable value; while the confirmatory factor analysis supports the proposal of an unifactorial structure with (GFI, Goodness of Fit Index 0.983; AGFI, Adjusted Goodness of Fit Index 0.978 and NFI, Normed Fit Index 0.962) [16]; also has criterion and concurrent validity, with Kappa concordance index values ​​of 0.66 (p<0.001) and 0.63 (p<0.001) respectively. The values ​​of these indices point in the same direction as those obtained from our instrument, resulting in adequate construct validity.

Regarding the Youth Risk Behavior Survey (YRBS), developed by the Centers for Disease Control and Prevention, it consists of 92–105 items, no studies have been conducted to determine its construct validity [18]; but when used in population studies, it is inferred that it has content validity; this is in accordance with the CCRAC, which through the Aiken V Index 0.96, indicates adequate content validity; and regarding its reliability, it was calculated by test-retest reliability, obtaining a kappa value of 61–100 % in three-quarters of the instrument’s items. YRBS has acceptable reliability, in the same way as CCRAC.

In Mexico, the NOM 047-SSA2-2015, For health care of the Age Group from 10 to 19 years of age [50], indicates that research must be done on risk behaviors in adolescence, emphasizing sexuality, drug use, accidents, violence, among others; and our CCRAC instrument coincides in its content validity with what is dictated by said norm. The screening instrument proposed by this regulation, HEAADSSS (for its acronym in English) [3], 50], there are no studies that determine its psychometric properties [3], however, being an instrument widely used in other regions, it is inferred that it has content validity.

Regarding the limitations of this study, the sample size used was the minimum necessary; however, there are recommendations that the minimum sample size should be 250 participants, in order to reduce the probability of a type 2 error [23], 24]. There are also suggestions that exploratory and confirmatory factor analyses be carried out in separate samples [25]. On the other hand, although it is not a common practice in psychometric studies, theoretically it is preferable for the sampling to be probability-based, thus increasing representativeness [19].

Conclusions

The 12-item Risk Behavior Questionnaire for Adolescents (Cuestionario de Conductas de Riesgo en la Adolescencia, CCRAC) was developed in Spanish and validated in adolescents of clinical context of tertiary mexican Pediatric Hospital. It is a brief, simple, and easy-to-score, free-use instrument that shows adequate psychometric properties in terms of reliability and validity. The items that comprise it are in line with epidemiological reports on risk behaviors whose presentation is of special interest in adolescence. The scores for the 12 items are Likert-type, and the severity is intuitive with a simple reading: 0 for absent behavior, one for present behavior, and two for present behavior with greater risk characteristics. Our instrument can be used as a questionnaire (present/absent) or as a scale (level of severity). Also, due to the characteristics of the instrument, it is feasible to use it in schools, primary care levels, or similar environments. This instrument is a contribution to increase the study and knowledge of risk behaviors at a stage of development that is especially vulnerable to these phenomena.

Future research is required to assess whether the psychometric properties remain consistent and replicable, especially in the general population; a high prevalence of risk behaviors was also reported, which merits further research and interventions related to public health (Table 1).