An analysis of publication trends of orthopedic surgery residency graduates in relation to academic achievement

Methods

Results

Of the 50 randomly selected orthopedic residency programs, 31 (62.0%) were included in our analysis, and 19 were excluded after a list of residency graduates were not obtained (Figure 1). Our sample included 341 orthopedic residency graduates, the majority of which were males (295, 86.5%) who graduated from an allopathic medical school (330, 96.8%) and of which went into private practice (264, 77.4%). Additional graduate characteristics can be found in Table 1. Altogether, these 341 graduates published a total of 1923 publications during the selected time interval, with a mean of 5.6 (SD, 7.2) publications per graduate. Graduates had a mean of 0.6 (SD, 2.1) publications before residency, 1.9 (SD, 2.8) during residency and 3.1 (SD, 5.5) after residency. Graduates who completed residency training in 2013 produced a total of 557 (mean, 5.4; SD, 6.6) publications, graduates from 2014 produced 700 (mean, 6.3; SD, 8.1) publications, and graduates from 2015 produced 666 (mean, 5.3; SD, 6.7) publications. The most common type of publications were observational studies (863/1923, 44.9%), and the most common journal to publish in was the Journal of Bone and Joint Surgery (n=120; Table 2). Graduates had a mean h-index of 3.2 (SD, 4.0).

Figure 1:

Program selection.

The Pearson correlation demonstrated a weak correlation between the number of publications obtained before and during residency (r=0.24) and between publications before and total publications after residency (r=0.21). Publishing research before residency was moderately correlated with an increase in h-index (r=0.45). Residents who pursued a fellowship had more publications (t₃₄₁=−3.0, p=0.003) and more first-author publications (t₃₄₁=−3.5, p=0.0128) than residents who did not pursue fellowship training. Residents with a higher h-index were more likely to have pursued a fellowship than those that did not participate in a fellowship (t₃₄₁=−3.0, p=0.003). A post-hoc analysis was performed utilizing a Pearson chi-squared test, which showed a significant relationship between gender and first-author publication after residency (X2=5.7, p < 0.05); 60% of women graduates were the first author on publications after residency compared to 42.0% of their male counterparts. Males and females entered into academia (66/295, 22.4%; 11/46, 23.9%, respectively), as well as fellowships (274/295, 92.9%; 43/46, 93.5%, exclusively), at similar rates. Residents that had a higher total number of first-author publications were more likely to enter academic medicine than those with lower total publications (t₃₄₁=−6.1, p < 0.001) (Table 3).

Our final logistic regression model examined the relationship between first-author publications before residency, fellowship training, and gender and whether the graduate went into academic medicine. No significant correlations were found for graduates who published one or more articles before residency (adjusted odds ratio [aOR], 2.1; 95% CI, 0.6–7.4). No significant correlations were observed between graduates pursuing fellowships and choosing to go into academic medicine (aOR, 1.1; 95% CI, 0.8–1.4). No significant correlations were found for gender (aOR, 0.9; 95% CI, 0.4–1.9) (Table 4).

Discussion

Results from our study suggest that the number of peer-reviewed publications produced by orthopedic surgery residents was associated with a higher likelihood of pursuing fellowship training and a higher likelihood to enter into academia. This finding is similar to studies of other surgical specialties [16, 17]. DeLong et al. [16] evaluated whether 624 plastic surgery residents who matched into fellowship training had an increased number of publications than residents who did not. Their analyses showed that research publications before and during residency correlated with an increase in both academic productivity and matching into fellowships [16]. Similarly, in the field of urology, 49 urology residents from the Mayo Clinic correlated a significant increase of those pursuing academic medicine with one or more publications prior to residency [17]. These findings among plastic surgery and urology residents coincide with our current analysis in orthopedic residents.

In the present study, we showed a mean of 5.6 publications per resident with a continuing increase from preresidency, during residency, and postresidency. Our results are consistent with previous studies, emphasizing the importance of research and matching into fellowships, specifically accepted publications in journals with high impact factors [18, 19]. A survey of 189 fellowship-matched orthopedic surgeons found on average 3.95 publications per fellow with 29.2% being in high-impact journals (American Journal of Sports Medicine [AJSM], Journal of Bone and Joint Surgery [American volume, JBJS Am], Journal of Shoulder and Elbow Surgery [JSES], or Arthroscopy) [18]. Another study of 130 orthopedic surgery residents found a mean number of total publications greater for those choosing a career in academia (4.8) than those who chose a nonacademic career (2.4) [19]. Taken together, these findings, when combined with ours, suggest that increased research productivity may be a necessary component for matching into fellowships and pursuit of a career in academic medicine.

We also compared several measures of research production by gender. In our study, females and males had a nearly equal number of publications, h-indices, and number of first-author publications. A rather interesting finding from our study was a significant relationship between gender and first-author publications postresidency. We found that 60.0% of the females in our sample were first authors after residency compared to only 42.0% of the males in our sample. A nonsignificant but important finding from our study is that male and female resident graduates entered into academics at nearly equal rates, 22.4% and 23.9% respectively. Ence et al. [20] found that, in orthopedic surgery, gender was not independently associated with an increased probability of senior academic rank. These authors also reported no statistically significant difference in m-indices (h-index divided by academic career duration in years) between men and women overall [20]. A recently published review comparing gender of orthopedic faculty found no significant difference in research productivity except at the assistant professor level [21]. Similar findings have been corroborated in a study comparing research productivity based on the gender of faculty in gastroenterology residencies [22]. A review of 34 different medical specialties comprising of 9952 academic physicians found no differences in research productivity throughout the careers of academic surgeons [23]. When comparing mean h-indices by gender, most surgical specialties have significant differences; one explanation for this finding is a significantly lower representation in surgical specialties rather than less research production [24]. The disproportionate percentage of total residency positions held by females in orthopedics has seen improvements, going from 11% in 2005 to 14.0% in 2016 [24]. Even after this increase, orthopedics still has the lowest proportion of female residents of any specialty. With a similar makeup in our sample (13.5% female), our findings add to the growing body of literature illustrating how little gender influences research production and academic success, specifically in surgical specialties.

Similarly, with the USMLE and COMLEX board examinations transitioning to a pass/fail grading system, research productivity may become an increasingly important component for applicants matching into orthopedic surgery residency [12, 25]. More than two-thirds of orthopedic program directors utilize USMLE Step 1 scores to screen the number of applicants necessary for review [26]. While the number of applicants placing into orthopedic surgery has remained steady over time—National Resident Matching Program (NRMP) match data suggest 80.0% placement from 2007 to 2014—abstracts reported by orthopedic applicants who matched more than doubled from 3.0 in 2007 to 6.7 in 2014 (p=0.04) [27]. Conversely, this number increased less drastically for all other specialty applicants, going from 2.2 in 2007 to 4.2 in 2014 (p=0.004) [27]. DePasse et al. [27] predicted a plateau in the mean USMLE scores, which we now see in the form of pass/fail scores beginning in January 2022. Additionally, another review utilizing NRMP data from 2006 to 2014 of matched US seniors was significantly greater than unmatched US seniors (p=0.035) for research production [28, 29]. For academic residency programs, particular research metrics may be useful during residency selection if the goal is to recruit trainees with more research experience who could translate this experience into independent lines of research. Future studies should evaluate the effect of this change on the research outcomes of medical students from a cohort of students before and after this change.

Our study has both strengths and limitations. Regarding its strengths, we performed all data extraction in duplicate with masked extractors. This practice is considered the gold standard for conducting systematic reviews and is recommended by the Cochrane Collaboration [30]. We also performed a random sample of orthopedic surgery residency programs to improve the generalizability of our findings. We had 31 of 50 residency programs in our sample, a response rate of 62.0%; thus, we feel as though our data accurately portray the current state of affairs with orthopedic surgery specifically pertaining to gender representation. We utilized a multi-step process for obtaining resident data, including contacting programs, which is more robust than what has been done in previous studies, which limited their data collection efforts to examining websites and excluding programs that did not have websites with the pertinent information. Regarding its limitations, we performed a cross-sectional study, and our results should be interpreted within the time frame of our sample and should not be generalized to other time periods. It is also possible that our random sample resulted in sampling bias. It is also possible that the surnames of residents may have changed over the course of the study, and we may have missed these changes during data collection. Our random generator includes few AOA-approved programs, potentially decreasing the representation of orthopedic graduates and academic achievement. Further investigation specifically for AOA-approved programs should be pursued.

Conclusions

Research productivity during residency training significantly correlates with the pursuit of academic achievement via fellowship or academic medicine. The present study represents an average of 5.6 publications, with the majority being observational reviews and published in high–impact-factor journals. Additionally, increased research output, especially first-author publications, significantly showed a positive correlation with further academic pursuits. Although increased research output showed further academic pursuits, 77.4% (n=264) of graduates pursued private practice vs 22.6% (n=77) entering academia. Contrary to the current literature, our study supports similar numbers of publications, h-indices, and number of first-author publications among women and men, thus illustrating that the only difference between men and women in orthopedic residencies is the rate at which they are accepted. This study provides a comprehensive appreciation of research productivity among orthopedic residents preresidency, during residency, and postresidency as it relates to additional academic pursuits, which may be useful for medical students and orthopedic residents in preparation for the application cycle.