Journal impact factor: the debate continues

Mario Plebani

doi:10.1515/cclm-2013-0549

Article Publicly Available

Journal impact factor: the debate continues

Mario Plebani

Published/Copyright: August 6, 2013

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information

From the journal Clinical Chemistry and Laboratory Medicine (CCLM) Volume 51 Issue 12

Champagne in hand to toast the new, boosted impact factor (IF) awarded to Clinical Chemistry and Laboratory Medicine (CCLM), I received an opinion paper by Eleftherios Diamandis [1] that adds fuel to the fiery debate on the usefulness and longevity of journal IF. As if in a nightmare, I was struggling with the idea of having devoted hours, and days of my life in the effort to improve a metric that may be useless, or even dangerous. The annual release of newly calculated IF is earnestly awaited by editors, publishers and potential authors. While editors celebrate an increase in their publication’s IF, particularly if significant, as in the case of CCLM this year, any decrease can send them into a huddle to figure out what has happened and work out the means to boost their ranking.

A citation-based metric, the IF is grounded on the average number of times a journal’s articles have been referenced in other articles [2]. Every year Thompson Reuters extracts the references from more than 9000 journals and calculates the IF for each of them by taking the number of citations to articles published in the journal in the previous 2 years and dividing this total by the number of articles published by the journal over the same period of time. It has been observed that this system for ranking scientific journals may fall prey to manipulation of the calculation and that it could be misused when ranking scientists by weighting their publications according to the IF of the journals their papers appear in [3]. The two criticisms, addressing somewhat different problems, point to the need for a different approach. Like any other algorithm, the one used to calculate IF is not a direct measure of quality, and any manipulation may affect the ratio between citations and articles published. For example, reviews are more frequently cited than primary research papers, and commentary articles may increase the self-citation rates. As stressed by Kai Simons, while the numerator is represented by all cited papers, “the (IF) denominator includes only primary research papers and reviews” [4]; this, tantamount to “Facta lex inventa fraus”, implies that the remedy lies not in finding a better algorithm but in calling editors and publishers to order with regard to their ethical behavior so as to prevent them from manipulating and artificially boosting the measure.

However, the number of citations and types of papers cited must remain a fundamental criterion for understanding the perspectives and the fields of interest of a journal’s readership. From this viewpoint, any criticism regarding the increased number of reviews published in our journal, e.g., must be weighed against the evidence given of their usefulness, and the appreciation expressed by the readership, as seen not only in their citations but also in questionnaires and interviews. For example, the papers most frequently cited in the CCLM are those that have contributed to the 2012 IF, and include a general review on the significance of the pre-analytical phase in quality of laboratory testing [5], an opinion paper on the diagnostic tests for improving the monitoring of new and emerging oral anticoagulants [6], and an intriguing article on statistical methods assessing the added usefulness of new biomarkers [7]. Indeed, we are proud of the publication of each of these papers, since they have all made a remarkable contribution by enhancing interest and knowledge in the field of diagnostic testing.

In my opinion, therefore, the former criticism should not affect the longevity of IF, as predicted by Diamandis, but lead to editors having a more ethical approach and to the scientific community developing and releasing guidelines and quality indicators that obviate the risk of fraud or cheating. In other words, when you play a match you have to accept the rules, although you should also strive to change them if they are wrong and, if you are successful, you should celebrate, just as we celebrated the new IF of CCLM. If you are defeated, you should work out how to improve your performance.

The latter criticism is more intriguing and “evidence-based” since it focuses on the misuse of IF as a measure of the quality of both individual research papers and individual researchers, a point raised almost 4 years ago in the journal [3]. This issue was addressed in the DORA (San Francisco Declaration on Research Assessment) declaration aiming to correct distortions in the evaluation of scientific research; the organization which aims to stop the use of the journal IF in judging an individual scientist’s work, has stated that the IF must not be used as “a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions” (www.ascb/SFdeclaration.htm).

Therefore, I completely agree with the DORA recommendations, which underline the need to:

eliminate the use of journal-based metrics, such as journal impact factors, in funding, appointment, and promotion considerations;
assess research on its own merits rather than on the basis of the journal in which the research is published;
capitalize on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).

Moreover, in my view it is undeniable that “there are no numerical shortcuts for evaluating research quality” [8]: even a surrogate measure such as the H-index has several drawbacks [9]. Ultimately quality is judged by scientists and, in the case of medicine, by patients and other stakeholders who may benefit from innovative diagnostic and therapeutic processes developed on the basis of new scientific insights.

On celebrating CCLM’s 50th anniversary [10], we stressed the need to circumvent current obstacles by maintaining a balance between assuring a valuable IF to the journal and honoring our commitment to provide the scientific community with reliable updates and usable information. I here reaffirm these goals and reiterate my thanks to all authors, reviewers, editorial board members, associate editors and the publisher for upholding the journal’s mission, and I wish our journal, Clinical Chemistry and Laboratory Medicine, further success.

References

1. Diamandis E. More discussion on journal impact factor. Clin Chem Lab Med, 2013;51:2271.10.1515/cclm-2013-0515Search in Google Scholar PubMed

2. Plebani M. Thank you, indeed! Clin Chem Lab Med 2011;49:1759–60.http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=000296772300001&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=b7bc2757938ac7a7a821505f8243d9f3Search in Google Scholar PubMed

3. Lippi G. The impact factor for evaluating scientists: the good, the bad and the ugly. Clin Chem Lab Med 2009;47:1585–6.http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=000272257900024&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=b7bc2757938ac7a7a821505f8243d9f3Search in Google Scholar PubMed

4. Simons K. The misused impact factor. Science 2008;322:165.10.1126/science.1165316Search in Google Scholar PubMed

5. Lippi G, Chance JJ, Church S, Dazzi P, Fontana R, Giavarina D, et al. Preanalytical quality improvement: from dream to reality. Clin Chem Lab Med 2011;49:1113–26.http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=000292538800004&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=b7bc2757938ac7a7a821505f8243d9f3Search in Google Scholar PubMed

6. Samama MM, Guinet C. Laboratory assessment of new anticoagulants. Clin Chem Lab Med 2011;49:761–72.http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=000289465700003&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=b7bc2757938ac7a7a821505f8243d9f3Search in Google Scholar PubMed

7. Pencina MJ, D’Agostino RB, Vasan RS. Statistical methods for assessment of added usefulness of new biomarkers. Clin Chem Lab Med 2010;48:1703–11.http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=000284624600004&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=b7bc2757938ac7a7a821505f8243d9f3Search in Google Scholar PubMed

8. Alberts B. Impact factor distortions. Science 2013;340:787.10.1126/science.1240319Search in Google Scholar PubMed

9. Lippi G, Mattiuzzi C. The challenges of evaluating scientists by H-index and citations in different biomedical research platforms. Clin Chim Acta 2013;421:57–8.10.1016/j.cca.2013.02.024 http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=000320220900010&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=b7bc2757938ac7a7a821505f8243d9f3Search in Google Scholar PubMed

10. Lippi G, Gillery P, Kazmierczak S, Lackner KJ, Melichar B, Siest G, et al. Clinical Chemistry and Laboratory Medicine: progress and new challenges for our 50-year-old journal. Clin Chem Lab Med 2013;51:5–7.http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=000312497600003&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=b7bc2757938ac7a7a821505f8243d9f310.1515/cclm-2012-0449Search in Google Scholar PubMed

Published Online: 2013-08-06

Published in Print: 2013-12-01

Articles in the same Issue

https://doi.org/10.1515/cclm-2013-0549