Reported completion of the USMLE Step 1 and match outcomes among senior osteopathic students in 2020

Methods

We performed a secondary analysis of specialty-level match data of the published match results from the 2020 National Residency Match Program (NRMP) for senior osteopathic students; the report is free and publicly available online [9]. This study does not meet the 2018 Revised Common Rule criteria for human subjects research because the data were not specifically collected for our study, and no one on our study team had access to any subject identifiers [10].

To mitigate any bias that more competitive specialties and those with fewer applicants would impose on the results, we restricted our analysis to the 10 specialties to which students applied most (i.e., those with the most applicants). The NRMP determined an applicant’s preferred specialty by their first-ranked program, and match success was defined as matching into the preferred specialty [9]. Because there is no incentive for withholding a failure or poor score from the NRMP on the optional research form, we assumed that senior osteopathic students who did not report taking the USMLE Step 1 did not take it. However, we also assumed that senior osteopathic students who did not report taking the COMLEX-USA Level 1 would not report taking the USMLE Step 1. Because these students were likely true nonreporters rather than individuals who did not take the USMLE Step 1, we subtracted the number of COMLEX-USA Level 1 nonreporters from the number of USMLE Step 1 nonreporters within each specialty. The remaining senior osteopathic students who did not report taking the USMLE Step 1 were assumed to have not taken the test.

We then determined the total number of senior osteopathic students that matched vs those that did not and stratified them by reported USMLE Step 1 completion within each specialty. Next, we reported the proportions matched with and without the USMLE Step 1. Then, we calculated odds ratios (ORs) for senior osteopathic students matching with the USMLE Step 1 compared to those without utilizing the Fisher’s exact test. Lastly, because students may not take the USMLE due to fear of performing poorly, we repeated the analysis with only senior osteopathic students who had reported USMLE Step 1 scores in ranges including or below the mean for their preferred specialty (Table 1). ORs with confidence intervals that did not include one were considered significant and all analyses were performed in RStudio^® (RStudio, PBC, Boston, MA, Version 2021.09.0+351).

Results

In 2020, 6,581 senior osteopathic students submitted rank order lists. A total of 6,055 (92%) of them gave consent for NRMP research [9]. The 10 specialties to which students applied most were Internal Medicine, Family Medicine, Emergency Medicine, Pediatrics, Psychiatry, Anesthesiology, General Surgery, Obstetrics and Gynecology, Diagnostic Radiology, and Orthopedic Surgery. The proportion of senior osteopathic students who reported taking the USMLE Step 1 ranged from 30.8 to 92.3% across the examined specialties (Table 2). The proportion of matched senior osteopathic students who reported taking the USMLE Step 1 was greater than the proportion who matched without the test across all 10 specialties (Table 2 and Figure 1). In addition, the proportion of matched senior osteopathic students who reported low USMLE Step 1 scores was greater than the proportion who matched without the test for all specialties except for Family Medicine and for Obstetrics and Gynecology (Table 2 and Figure 1).

Figure 1:

A dumbbell plot comparing the proportions of matched senior osteopathic students by reported USMLE Step 1 completion status stratified by specialty.

Nevertheless, reported completion of the USMLE Step 1 was associated with matching for those who applied to Internal Medicine (OR 3.3 [95% confidence interval 2.07 to 5.48]), Emergency Medicine (2.1 [1.35 to 3.17]), Pediatrics (4.4 [1.38 to 18.63]), Psychiatry (2.5 [1.34 to 4.98]), Anesthesiology (3.4 [1.57 to 7.32]), and General Surgery (3.1 [1.56 to 6.14]) (Table 3). After repeating the analysis with only senior osteopathic students who reported low USMLE Step 1 scores, the association remained significant for those who applied to Internal Medicine (2.5 [1.49 to 4.28]), Anesthesiology (2.6 [1.17 to 5.54]), and General Surgery (2.5 [1.24 to 5.04]) (Table 3).

Discussion

In this secondary analysis of published 2020 NRMP match results for senior osteopathic students who applied to the 10 most popular specialties, we found that reported completion of the USMLE Step 1 was associated with matching for those who applied to Internal Medicine, Emergency Medicine, Pediatrics, Psychiatry, Anesthesiology, and General Surgery. Furthermore, after limiting the analysis to those with low USMLE Step 1 scores, this association remained significant for those who applied to Internal Medicine, Anesthesiology, and General Surgery (Table 3).

These results are consistent with previous work. In a study of 350 osteopathic applicants to emergency medicine residency programs from the 2010 to 2011 application season, those who took the USMLE were more likely to match than those who did not take it, 126/208 (61%) vs 55/142 (39%), respectively [11]. In addition, performance on the USMLE Step 1 has been associated with matching into preferred specialties for osteopathic applicants [12].

Interestingly, the association between taking the USMLE Step 1 with matching was inconsistent between specialties (Table 3). For example, reported completion of the USMLE Step 1 was not associated with matching for those who applied to Family Medicine, Obstetrics and Gynecology, Diagnostic Radiology, and Orthopedic Surgery (Table 3). Therefore, completing the USMLE Step 1 may not improve match outcomes among all specialties. However, reported completion of the USMLE Step 1 did not appear to harm match outcomes across the examined specialties, even with a low score (Table 3). Since most students change their specialty pursuits during medical school [13, 14], the association between completion of the USMLE Step 1 and matching for multiple specialties may outweigh the risk of performing poorly on the test for many osteopathic students.

However, the USMLE Step 1 and COMLEX-USA Level 1 are transitioning to pass/fail score reports [15, 16]. It is unknown how this change will alter the value of other aspects of the medical student application [17, 18]. However, osteopathic students will likely benefit from this change because it removes the necessity to obtain a competitive score on the USMLE Step 1, therefore lowering the threshold to take the test. Furthermore, as our results suggest, senior osteopathic students who reported taking the USMLE Step 1 were more likely to match into several specialties regardless of score (Table 2).

Conclusions

Among senior osteopathic students who applied to the 10 most popular specialties in the 2020 NRMP Match, reported completion of the USMLE Step 1 was associated with matching for those who applied to Internal Medicine, Emergency Medicine, Pediatrics, Psychiatry, Anesthesiology, and General Surgery. After limiting the analysis to senior osteopathic students with low USMLE Step 1 scores, this association remained significant for those who applied to Internal Medicine, Anesthesiology, and General Surgery.