Investigating the association between school substance programs and student substance use: accounting for informative cluster size

Aya A. Mitani; Yushu Zou; Scott T. Leatherdale; Karen A. Patte

doi:10.1515/em-2024-0028

Article

Investigating the association between school substance programs and student substance use: accounting for informative cluster size

Aya A. Mitani , Yushu Zou , Scott T. Leatherdale and Karen A. Patte

Published/Copyright: August 26, 2025

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information

From the journal Epidemiologic Methods Volume 14 Issue 1

Abstract

Objectives

The use of substances in adolescents is an increasing public health problem. Many high schools in Canada have implemented school-based programs to mitigate student substance use, but their utility is not conclusive. Polysubstance use data collected on students from multiple schools may be subject to informative cluster size (ICS). The objective of this study was to investigate whether a multivariate analysis approach that addresses ICS provides different conclusions from univariate analyses and methods that do not account for ICS.

Methods

We used data from the 2018/2019 cycle of the Cannabis, Obesity, Mental health, Physical activity, Alcohol, Smoking, and Sedentary Behaviour (COMPASS) study, an ongoing prospective cohort study that annually collects data from Canadian high schools and students. We compared results from four analytical approaches that estimate marginal associations between each school substance program and the four substance use behaviours (binge drinking, cannabis, e-cigarette, and cigarette): univariate generalized estimating equations (GEE), univariate cluster-weighted GEE (CWGEE), multivariate GEE, and multivariate CWGEE.

Results

We observed that the proportion of students who engage in each of the four behaviours was higher in small schools and lower in large schools. In general, the univariate and multivariate analyses produced comparable results. Some differences existed between multivariate CWGEE and GEE. CWGEE indicated that the school program on cannabis had an odds ratio (OR) and 95 % confidence interval (CI) of 0.83 (0.73, 0.95) on all substance use, but GEE produced a null association with an OR (95 % CI) of 0.92 (0.79, 1.07).

Conclusions

When ICS is present in clustered school data, weighted and unweighted analyses may produce different results. Care is needed to investigate the relationship between cluster size and the outcome, and use appropriate methods for analysis. Certain substance programs may influence student behaviour in other substances, highlighting the need for a multivariate analytical approach when studying the use of substances by adolescents.

Keywords: adolescent health; clustered data; cluster-weighted generalized estimating equations; marginal models; multivariate analysis

Corresponding author: Aya A. Mitani, Assistant Professor, Dalla Lana School of Public Health, University of Toronto, 155 College Street, Toronto, Ontario, M5T 3M7, Canada, E-mail: aya.mitani@utoronto.ca

Funding source: Natural Sciences and Engineering Research Council of Canada

Award Identifier / Grant number: RGPIN-2022-05356

Funding source: Institute of Population and Public Health

Award Identifier / Grant number: MOP-114875

Research ethics: The University of Waterloo Office of Research Ethics and appropriate School Board committees approved all procedures for the COMPASS study. Secondary analysis of COMPASS data was approved by the University of Toronto Research Ethics Boards (RIS Human Protocol Number 44375).
Informed consent: Informed consent was obtained from all individuals included in this study, or their legal guardians or wards.
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission. A.M. conceptualized the study, conducted the literature review, contributed to data interpretation, wrote the first draft of the manuscript and contributed to editing the manuscript. Y.Z. conducted the literature review, cleaned the data, conducted data analysis, contributed to data interpretation, wrote the first draft of the manuscript, and contributed to editing the manuscript. SL contributed to the review and editing of the manuscript, planned and obtained funding for the data collection, and conceptualized the larger cost study. K.P. contributed to the review and editing of the manuscript and planned and obtained funding for the data collection. All authors supported the discussion, interpreted the data, critically reviewed the manuscript, and approved the final manuscript.
Use of Large Language Models, AI and Machine Learning Tools: None declared.
Conflict of interest: All other authors state no conflict of interest.
Research funding: The COMPASS study has been supported by a bridge grant from the CIHR Institute of Nutrition, Metabolism and Diabetes (INMD) through the “Obesity – Interventions to Prevent or Treat” priority funding awards (OOP-110788; awarded to S.L.), an operating grant from the CIHR Institute of Population and Public Health (IPPH) (MOP-114875; awarded to S.L.), a CIHR project grant (PJT-148562; awarded to S.L.), a CIHR bridge grant (PJT-149092; awarded to K.P./S.L.), a CIHR project grant (PJT-159693; awarded to K.P.), and by a research funding arrangement with Health Canada (\#1617-HQ-000012; contract awarded to S.L.), a CIHR-Canadian Centre on Substance Use and Addiction (CCSA) team grant (OF7 B1-PCPEGT 410-10-9633; awarded to S.L.), a project grant from the CIHR Institute of Population and Public Health (IPPH) (PJT-180262; awarded to S.L. and K.P.). A SickKids Foundation New Investigator Grant, in partnership with CIHR Institute of Human Development, Child and Youth Health (IHDCYH) (Grant No. NI21-1193; awarded to K.P.) funds a mixed methods study examining the impact of the COVID-19 pandemic on youth mental health, leveraging COMPASS study data. The COMPASS-Quebec project additionally benefits from funding from the Ministère de la Santé et des Services sociaux of the province of Québec, and the Direction régionale de santé publique du CIUSSS de la Capitale-Nationale. This research conducted for this paper was partially supported by the Natural Sciences and Engineering Research Council of Canada - Discovery Grants Program (RGPIN-2022-05356; awarded to A.M.). K.P. is supported by the Canada Research Chairs program.
Data availability: COMPASS study data are available upon request through completion and approval of an online form: https://uwaterloo.ca/compass-system/information-researchers/data-usage-application.

References

1. Hopfer, C. Implications of marijuana legalization for adolescent substance use. Subst Abuse 2014;35:331–5. https://doi.org/10.1080/08897077.2014.943386.Search in Google Scholar PubMed PubMed Central

2. Lim, CC, Sun, T, Leung, J, Chung, JY, Gartner, C, Connor, J, et al.. Prevalence of adolescent cannabis vaping: a systematic review and meta-analysis of US and Canadian studies. JAMA Pediatr 2022;176:42–51. https://doi.org/10.1001/jamapediatrics.2021.4102.Search in Google Scholar PubMed PubMed Central

3. Alarcó-Rosales, R, Sánchez-SanSegundo, M, Ferrer-Cascales, R, Albaladejo-Blazquez, N, Lordan, O, Zaragoza-Martí, A. Effects of a school-based intervention for preventing substance use among adolescents at risk of academic failure: a pilot study of the reasoning and rehabilitation V2 program. Healthcare 2021;9:1488. https://doi.org/10.3390/healthcare9111488.Search in Google Scholar PubMed PubMed Central

4. Evans-Whipp, T, Beyers, JM, Lloyd, S, Lafazia, AN, Toumbourou, JW, Arthur, MW, et al.. A review of school drug policies and their impact on youth substance use. Health Promot Int 2004;19:227–34. https://doi.org/10.1093/heapro/dah210.Search in Google Scholar PubMed

5. Burnett, T, Battista, K, Butt, M, Sherifali, D, Leatherdale, ST, Dobbins, M. The association between public health engagement in school-based substance use prevention programs and student alcohol, cannabis, e-cigarette and cigarette use. Can J Public Health 2023;114:94–103. https://doi.org/10.17269/s41997-022-00655-3.Search in Google Scholar PubMed PubMed Central

6. Williams, GC, Cole, AG, de Groh, M, Jiang, Y, Leatherdale, ST. More support needed: evaluating the impact of school e-cigarette prevention and cessation programs on e-cigarette initiation among a sample of Canadian secondary school students. Prev Med 2022;155:106924. https://doi.org/10.1016/j.ypmed.2021.106924.Search in Google Scholar PubMed

7. Liu, XQ, Guo, YX, Wang, X. Delivering substance use prevention interventions for adolescents in educational settings: a scoping review. World J Psychiatr 2023;13:409. https://doi.org/10.5498/wjp.v13.i7.409.Search in Google Scholar PubMed PubMed Central

8. Yang, Y, Butt, ZA, Leatherdale, ST, Morita, PP, Wong, A, Rosella, L, et al.. Exploring the dynamic transitions of polysubstance use patterns among Canadian youth using latent Markov models on compass data. Lancet Reg Health – Am 2022;16:100389. https://doi.org/10.1016/j.lana.2022.100389.Search in Google Scholar PubMed PubMed Central

9. Fagan, MJ, Duncan, MJ, Bedi, RP, Puterman, E, Leatherdale, ST, Faulkner, G. The prospective association between physical activity and initiation of current substance use among adolescents: examining the role of school connectedness. Ment Health Phys Act 2023;24:100503. https://doi.org/10.1016/j.mhpa.2023.100503.Search in Google Scholar

10. Fagan, MJ, Duncan, MJ, Bedi, RP, Puterman, E, Leatherdale, ST, Faulkner, G. Physical activity and substance use among Canadian adolescents: examining the moderating role of school connectedness. Front Public Health 2022;10:889987. https://doi.org/10.3389/fpubh.2022.889987.Search in Google Scholar PubMed PubMed Central

11. Williams, GC, Burns, KE, Battista, K, de Groh, M, Jiang, Y, Leatherdale, ST. High school intramural participation and substance use: a longitudinal analysis of COMPASS data. Subst Use Misuse 2021;56:1108–18. https://doi.org/10.1080/10826084.2021.1901932.Search in Google Scholar PubMed

12. Kristjansson, AL, Sigfusdottir, ID, Allegrante, JP. Adolescent substance use and peer use: a multilevel analysis of cross-sectional population data. Subst Abuse Treat Prev Pol 2013;8:1–10. https://doi.org/10.1186/1747-597x-8-27.Search in Google Scholar

13. Doggett, A, Godin, KM, Schell, O, Wong, SL, Jiang, Y, Leatherdale, ST. Assessing the impact of sports and recreation facility density within school neighbourhoods on Canadian adolescents’ substance use behaviours: quasi-experimental evidence from the COMPASS study, 2015–2018. BMJ open 2021;11:e046171. https://doi.org/10.1136/bmjopen-2020-046171.Search in Google Scholar PubMed PubMed Central

14. Williams, GC, Burns, KE, Battista, K, de Groh, M, Jiang, Y, Leatherdale, ST. High school sport participation and substance use: a cross-sectional analysis of students from the COMPASS study. Addict Behav Rep 2020;12:100298. https://doi.org/10.1016/j.abrep.2020.100298.Search in Google Scholar PubMed PubMed Central

15. Seaman, S, Pavlou, M, Copas, A. Review of methods for handling confounding by cluster and informative cluster size in clustered data. Statistics Med 2014;33:5371–87. https://doi.org/10.1002/sim.6277.Search in Google Scholar PubMed PubMed Central

16. Kahan, BC, Li, F, Copas, AJ, Harhay, MO. Estimands in cluster-randomized trials: choosing analyses that answer the right question. Int J Epidemiol 2022;52:107–18. https://doi.org/10.1093/ije/dyac131.Search in Google Scholar PubMed PubMed Central

17. Hoffman, EB, Sen, PK, Weinberg, CR. Within-cluster resampling. Biometrika 2001;88:1121–34. https://doi.org/10.1093/biomet/88.4.1121.Search in Google Scholar

18. Williamson, JM, Datta, S, Satten, GA. Marginal analyses of clustered data when cluster size is informative. Biometrics 2003;59:36–42. https://doi.org/10.1111/1541-0420.00005.Search in Google Scholar PubMed

19. O’Malley, PM, Johnston, LD, Bachman, JG, Schulenberg, JE, Kumar, R. How substance use differs among American secondary schools. Prev Sci 2006;7:409–20. https://doi.org/10.1007/s11121-006-0050-5.Search in Google Scholar PubMed

20. National Center on Addiction and Substance Abuse at Columbia University. National survey of American attitudes on substance abuse VIII: teens and parents. New York, NY: Columbia University; 2003.Search in Google Scholar

21. Mitani, AA, Kaye, EK, Nelson, KP. Marginal analysis of multiple outcomes with informative cluster size. Biometrics 2021;77:271–82. https://doi.org/10.1111/biom.13241.Search in Google Scholar PubMed PubMed Central

22. Leatherdale, ST, Brown, KS, Carson, V, Childs, RA, Dubin, JA, Elliott, SJ, et al.. The COMPASS study: a longitudinal hierarchical research platform for evaluating natural experiments related to changes in school-level programs, policies and built environment resources. BMC Public Health 2014;14:1–7. https://doi.org/10.1186/1471-2458-14-331.Search in Google Scholar PubMed PubMed Central

23. White, VM, Hill, DJ, Effendi, Y. How does active parental consent influence the findings of drug-use surveys in schools? Eval Rev 2004;28:246–60. https://doi.org/10.1177/0193841x03259549.Search in Google Scholar PubMed

24. Bond, L, Butler, H, Thomas, L, Carlin, J, Glover, S, Bowes, G, et al.. Social and school connectedness in early secondary school as predictors of late teenage substance use, mental health, and academic outcomes. J Adolesc Health 2007;40:357-e9. https://doi.org/10.1016/j.jadohealth.2006.10.013.Search in Google Scholar PubMed

25. McNeely, CA, Nonnemaker, JM, Blum, RW. Promoting school connectedness: evidence from the national longitudinal study of adolescent health. J Sch Health 2002;72:138–46. https://doi.org/10.1111/j.1746-1561.2002.tb06533.x.Search in Google Scholar PubMed

26. Weatherson, KA, O’Neill, M, Lau, EY, Qian, W, Leatherdale, ST, Faulkner, GEJ. The protective effects of school connectedness on substance use and physical activity. J Adolesc Health 2018;63:724–31. https://doi.org/10.1016/j.jadohealth.2018.07.002.Search in Google Scholar PubMed

27. Chaganty, NR. An alternative approach to the analysis of longitudinal data via generalized estimating equations. J Stat Plann Inference 1997;63:39–54. https://doi.org/10.1016/s0378-3758-96-00203-0.Search in Google Scholar

28. Akre, C, Michaud, PA, Berchtold, A, Suris, JC. Cannabis and tobacco use: where are the boundaries? A qualitative study on cannabis consumption modes among adolescents. Health Educ Res 2010;25:74–82. https://doi.org/10.1093/her/cyp027.Search in Google Scholar PubMed

Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/em-2024-0028).

Received: 2024-11-15

Accepted: 2025-07-31

Published Online: 2025-08-26

You are currently not able to access this content.

Articles in the same Issue

https://doi.org/10.1515/em-2024-0028

Keywords for this article

adolescent health; clustered data; cluster-weighted generalized estimating equations; marginal models; multivariate analysis