Home Using specific, validated vs. non-specific, non-validated tools to measure a subjective concept: application on COVID-19 burnout scales in a working population
Article
Licensed
Unlicensed Requires Authentication

Using specific, validated vs. non-specific, non-validated tools to measure a subjective concept: application on COVID-19 burnout scales in a working population

  • Chadia Haddad ORCID logo EMAIL logo , Aline Hajj ORCID logo , Hala Sacre ORCID logo , Rony M. Zeenny ORCID logo , Marwan Akel ORCID logo , Katia Iskandar ORCID logo and Pascale Salameh ORCID logo
Published/Copyright: September 6, 2024
Become an author with De Gruyter Brill
Submit Manuscript Author Information
Epidemiologic Methods
From the journal Epidemiologic Methods Volume 13 Issue 1

Abstract

Objectives

The first objective is to compare the psychometric properties of two scales, measuring COVID-19-related burnout in a general working population during an economic crisis. The second objective is to compare the relevance through the assessment of statistically significant associations between the independent variables and the validated (scale 1) or non-validated (scale 2) scales taken as dependent variables.

Methods

This study enrolled 151 Lebanese participants, using a snowball sampling method. Two scales that measure burnout during COVID-19 were used.

Results

A significantly strong correlation was found between the validated COVID-19 burnout scale (scale 1) and the new pandemic-related burnout scale (scale 2) (r=0.796, p<0.001). A first linear regression on scale 1 (dependent) showed that increased concern about the impact of the economic crisis and COVID-19 (Beta=9.61) was significantly associated with higher COVID-19 burnout. However, higher financial well-being (Beta=−0.23) and working as a full timer (Beta=−7.80) were significantly associated with a lower COVID-19 burnout score. A second regression model on scale 2 (dependent) showed that higher financial well-being was only significantly associated with a lower pandemic-related burnout score (Beta=−0.72).

Conclusions

Our results showed that more specific scales have better psychometric properties while using non-validated, non-specific scales to evaluate an outcome might lead to biased associations and incorrect conclusions.

Keywords: scale validation; reliability; psychometric properties; burnout; pandemic; COVID-19
Corresponding author: Chadia Haddad, PhD, INSPECT-LB (Institut National de Santé Publique, d’Épidémiologie Clinique et de Toxicologie-Liban), Beirut, Lebanon; School of Medicine, Lebanese American University, Byblos, Lebanon; Research Department, Psychiatric Hospital of the Cross, P.O. Box 60096, Jal el Dib, Lebanon; and School of Health Sciences, Modern University for Business and Science, Beirut, Lebanon, E-mail:

Chadia Haddad and Aline Hajj First Co-authors.

Acknowledgments

The authors would like to thank all those who participated in this study by filling up and spreading the web-based online survey.

  1. Research ethics: The Research and Ethics Committee at the Lebanese International University School of Pharmacy approved the study protocol (2020RC-056-LIUSOP). All methods were carried out in accordance with relevant guidelines and regulations.

  2. Informed consent: An online informed consent was obtained from all subjects and/or their legal guardian(s).

  3. Author contributions: PS designed the study; PS, AH, CH drafted the manuscript; PS and CH carried out the analysis and interpreted the results; HS, RZ, MA, KI assisted in drafting and reviewing the manuscript; PS supervised the course of the article, HS revised and edited the article edited for English language. All authors reviewed and approved the final version of the manuscript.

  4. Competing interests: The authors states no conflict of interest.

  5. Research funding: None declared.

  6. Data availability: The raw data can be obtained on request from the corresponding author.

References

1. ThoughtCo. The differences between indexes and scales; 2019. Available from: https://www.thoughtco.com/indexes-and-scales-3026544 [Accessed 02 Sept 2021].Search in Google Scholar

2. DeVellis, RF, Thorpe, CT. Scale development: theory and applications. Chapel Hill: The University of North Carolina at Chapel Hill-Sage Publications; 2022.Search in Google Scholar

3. Souza, ACD, Alexandre NMC, Guirardello EDB, Alexandre NMC. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol Serv Saude 2017;26:649–59. https://doi.org/10.5123/s1679-49742017000300022.Search in Google Scholar

4. Dima, AL. Scale validation in applied health research: tutorial for a 6-step R-based psychometrics protocol. Health Psychol Behav Med 2018;6:136–61. https://doi.org/10.1080/21642850.2018.1472602.Search in Google Scholar PubMed PubMed Central

5. Tsang, S, Royse, CF, Terkawi, AS. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J Anaesth 2017;11:S80. https://doi.org/10.4103/sja.sja_203_17.Search in Google Scholar

6. Elangovan, N, Sundaravel, E. Method of preparing a document for survey instrument validation by experts. MethodsX 2021;8:101326. https://doi.org/10.1016/j.mex.2021.101326.Search in Google Scholar PubMed PubMed Central

7. Fries, JF, Krishnan, E, Rose, M, Lingala, B, Bruce, B. Improved responsiveness and reduced sample size requirements of PROMIS physical function scales with item response theory. Arthritis Res Ther 2011;13:1–8. https://doi.org/10.1186/ar3461.Search in Google Scholar PubMed PubMed Central

8. Morgado, FF, Meireles, JF, Neves, CM, Amaral, A, Ferreira, ME. Scale development: ten main limitations and recommendations to improve future research practices. Psicol Reflexão Crítica 2017;30. https://doi.org/10.1186/s41155-016-0057-1.Search in Google Scholar PubMed PubMed Central

9. Hogan, TP, Agnello, J. An empirical study of reporting practices concerning measurement validity. Educ Psychol Meas 2004;64:802–12. https://doi.org/10.1177/0013164404264120.Search in Google Scholar

10. Bramstedt, KA. The carnage of substandard research during the COVID-19 pandemic: a call for quality. J Med Ethics 2020a;46:803–7. https://doi.org/10.1136/medethics-2020-106494.Search in Google Scholar PubMed

11. Shah, K, Charan, J, Sinha, A, Saxena, D. Retraction rates of research articles addressing COVID-19 pandemic: is it the evolving COVID epidemiology or scientific misconduct? Indian J Community Med 2021;46:352–4. https://doi.org/10.4103/ijcm.IJCM_732_20.Search in Google Scholar PubMed PubMed Central

12. Bramstedt, KA. Luxembourg’s approach to research integrity during the COVID-19 pandemic. Acc Res 2020b;27:396–400. https://doi.org/10.1080/08989621.2020.1778473.Search in Google Scholar PubMed

13. Safiye, T, Gutić, M, Milidrag, A, Zlatanović, M, Radmanović, B. The impact of COVID-19 on mental health: the protective role of resilience and capacity for mentalizing. London. IntechOpen; 2022. https://doi.org/10.5772/intechopen.106161.Search in Google Scholar

14. Kaye, AD, Okeagu, CN, Pham, AD, Silva, RA, Hurley, JJ, Arron, BL, et al.. Economic impact of COVID-19 pandemic on health care facilities and systems: international perspectives. Best Pract Res Clin Anaesthesiol 2020;35:293–306. https://doi.org/10.1016/j.bpa.2020.11.009.Search in Google Scholar PubMed PubMed Central

15. Noy, I, Doan, N, Ferrarini, B, Park, D. Measuring the economic risk of COVID‐19. Glob Policy 2020;11:413–23. https://doi.org/10.1111/1758-5899.12851.Search in Google Scholar

16. Tan, BY, Kanneganti, A, Lim, LJ, Tan, M, Chua, YX, Tan, L, et al.. Burnout and associated factors among health care workers in Singapore during the COVID-19 pandemic. J Am Med Dir Assoc 2020;21:1751–8. https://doi.org/10.1016/j.jamda.2020.09.035.Search in Google Scholar PubMed PubMed Central

17. Yıldırım, M, Arslan, G, Wong, PT. Meaningful living, resilience, affective balance, and psychological health problems among Turkish young adults during coronavirus pandemic. Curr Psychol 2022;41:7812–23.10.1007/s12144-020-01244-8Search in Google Scholar PubMed PubMed Central

18. World Health Organization (WHO). Burn-out an “occupational phenomenon”: international classification of diseases; 2019. Available from: https://www.who.int/news/item/28-05-2019-burn-out-an-occupational-phenomenon-international-classification-of-diseases [Accessed 25 Apr 2023].Search in Google Scholar

19. Masand, P, Patkar, A, Tew, C, Hoerner, A, Szabo, ST, Gupta, S. Mental health and COVID-19: challenges and multimodal clinical solutions. J Psychiatr Pract 2021;27:254–64. https://doi.org/10.1097/PRA.0000000000000560.Search in Google Scholar PubMed PubMed Central

20. Mollica, RF, Fernando, DB, Augusterfer, EF. Beyond burnout: responding to the COVID-19 pandemic challenges to self-care. Curr Psychiatry Rep 2021;23:21. https://doi.org/10.1007/s11920-021-01230-2.Search in Google Scholar PubMed PubMed Central

21. Baxter, P. Invalid measurement validity. Dev Med Child Neurol 2005;47:291. https://doi.org/10.1111/j.1469-8749.2005.tb01137.x.Search in Google Scholar

22. Price, PC, Jhangiani, RS, Chiang, I-CA. Chapter 5: psychological measurement. Reliability and validity of measurement. In: Research methods in psychology - 2nd canadian edition. Canada: BCcampus Open Education; 2015.Search in Google Scholar

23. Yıldırım, M, Solmaz, F. COVID-19 burnout, COVID-19 stress and resilience: initial psychometric properties of COVID-19 Burnout Scale. Death Stud 2020:1–9. https://doi.org/10.1080/07481187.2020.1818885.Search in Google Scholar PubMed

24. Khasne, RW, Dhakulkar, BS, Mahajan, HC, Kulkarni, AP. Burnout among healthcare workers during COVID-19 pandemic in India: results of a questionnaire-based survey. Indian J Crit Care Med 2020;24:664.10.5005/jp-journals-10071-23518Search in Google Scholar PubMed PubMed Central

25. Rodríguez-López, AM, Rubio-Valdehita, S, Díaz-Ramiro, EM. Influence of the COVID-19 pandemic on mental workload and burnout of fashion retailing workers in Spain. Int J Environ Res Publ Health 2021;18:983. https://doi.org/10.3390/ijerph18030983.Search in Google Scholar PubMed PubMed Central

26. Shanafelt, TD, West, CP, Sinsky, C, Trockel, M, Tutty, M, Satele, DV, et al.. Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2017. Mayo Clin Proc 2019;94:1681–94.10.1016/j.mayocp.2018.10.023Search in Google Scholar PubMed

27. Prawitz, A, Garman, ET, Sorhaindo, B, O’Neill, B, Kim, J, Drentea, P. InCharge financial distress/financial well-being scale: development, administration, and score interpretation. J Financ Couns Plan 2006;17.10.1037/t60365-000Search in Google Scholar

28. Malach-Pines, A. The burnout measure, short version. Int J Stress Manag 2005;12:78. https://doi.org/10.1037/1072-5245.12.1.78.Search in Google Scholar

29. Moron, M, Yildirim, M, Jach, L, Nowakowska, J, Atlas, K. Exhausted due to the pandemic: validation of coronavirus stress measure and COVID-19 burnout scale in a polish sample. Curr Psychol 2021:1–10. https://doi.org/10.1007/s12144-021-02543-4.Search in Google Scholar PubMed PubMed Central

30. Lau, SS, Ho, CC, Pang, RC, Su, S, Kwok, H, Fung, S-f., et al.. Measurement of burnout during the prolonged pandemic in the Chinese zero-COVID context: COVID-19 burnout views scale. Front Public Health 2022;10:1039450. https://doi.org/10.3389/fpubh.2022.1039450.Search in Google Scholar PubMed PubMed Central

31. Boateng, GO, Neilands, TB, Frongillo, EA, Melgar-Quinonez, HR, Young, SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health 2018;6:149. https://doi.org/10.3389/fpubh.2018.00149.Search in Google Scholar PubMed PubMed Central

32. Safiye, T, Vukčević, B, Gutić, M, Milidrag, A, Dubljanin, D, Dubljanin, J, et al.. Resilience, mentalizing and burnout syndrome among healthcare workers during the COVID-19 pandemic in Serbia. Int J Environ Res Publ Health 2022;19:6577. https://doi.org/10.3390/ijerph19116577.Search in Google Scholar PubMed PubMed Central

33. Althubaiti, A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc 2016;9:211–17. https://doi.org/10.2147/JMDH.S104807.Search in Google Scholar PubMed PubMed Central

34. World Bank Group. Lebanon’s economic update – april 2020; 2020. Available from: https://www.worldbank.org/en/country/lebanon/publication/economic-update-april-2020 [Accessed 25 May 2020].Search in Google Scholar

35. Al Hariri, M, Hamade, B, Bizri, M, Salman, O, Tamim, H, Al Jalbout, N. Psychological impact of COVID-19 on emergency department healthcare workers in a tertiary care center during a national economic crisis. Am J Emerg Med 2022;51:342–7. https://doi.org/10.1016/j.ajem.2021.10.055.Search in Google Scholar PubMed PubMed Central

36. Anthoine, E, Moret, L, Regnault, A, Sébille, V, Hardouin, J-B. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcome 2014;12:1–10. https://doi.org/10.1186/s12955-014-0176-2.Search in Google Scholar PubMed PubMed Central

Received: 2024-04-30
Accepted: 2024-08-17
Published Online: 2024-09-06

© 2024 Walter de Gruyter GmbH, Berlin/Boston

Articles in the same Issue

  1. Research Articles
  2. Using specific, validated vs. non-specific, non-validated tools to measure a subjective concept: application on COVID-19 burnout scales in a working population
  3. Linked shrinkage to improve estimation of interaction effects in regression models
  4. A study of a deterministic model for meningitis epidemic
  5. Population dynamic study of two prey one predator system with disease in first prey using fuzzy impulsive control
  6. Leveraging data from multiple sources in epidemiologic research: transportability, dynamic borrowing, external controls, and beyond
  7. Temporal discontinuity trials and randomization: success rates versus design strength
  8. Effect of designations of index date in externally controlled trials: an empirical example
https://doi.org/10.1515/em-2024-0013
Keywords for this article
scale validation; reliability; psychometric properties; burnout; pandemic; COVID-19

Articles in the same Issue

  1. Research Articles
  2. Using specific, validated vs. non-specific, non-validated tools to measure a subjective concept: application on COVID-19 burnout scales in a working population
  3. Linked shrinkage to improve estimation of interaction effects in regression models
  4. A study of a deterministic model for meningitis epidemic
  5. Population dynamic study of two prey one predator system with disease in first prey using fuzzy impulsive control
  6. Leveraging data from multiple sources in epidemiologic research: transportability, dynamic borrowing, external controls, and beyond
  7. Temporal discontinuity trials and randomization: success rates versus design strength
  8. Effect of designations of index date in externally controlled trials: an empirical example
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 27.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/em-2024-0013/html
Scroll to top button