Research Integrity: Science Community Needs to Address Threats by Research Assessments

The Principle of Universality (freedom and responsibility) of Science

The free and responsible practice of science is fundamental to scientific advancement and human and environmental well-being. Such practice, in all its aspects, requiresfreedom of movement, association, expression and communication for scientists, as well as equitable access to data, information, and other resources for research. It requires responsibility at all levels to carry out and communicate scientific work with integrity, respect, fairness, trustworthiness, and transparency, recognising its benefits and possible harms.

In advocating the free and responsible practice of science, ICSU promotes equitable opportunitiesfor access to science and its benefits, and opposes discrimination based on such factors as ethnic origin, religion, citizenship, language, political or other opinion, sex, gender identity, sexual orientation, disability, or age.

www.icsu.org/freedom-responsibility/cfrs/statute-5

ICSU’s Promotion of Research Integrity

The raising of awareness for and the promotion of research integrity is part of the brief of the ICSU Committee on Freedom and Responsibility in the conduct of Science (CFRS), the guardian of the Universality of Science Principle. This element of the Committee’s work, documented on the ICSU website’s “Freedom & Responsibility Portal,”(1) comprised co-sponsoring the three editions of the World Conferences on Research Integrity (WCRI) so far (Lisbon 2007, Singapore 2010, Montreal 2013). With the adoption of the “Singapore Statement on Research Integrity,” (2) the conference in Singapore in 2010 was a significant achievement and success, also for CFRS and ICSU: the 230 participants endorsed this consensus document on globally applicable principles and responsibilities related to research integrity and scientific conduct. Based on last year’s WCRI in Montreal, a statement on research integrity principles applied to cross-boundary research collaborations was adopted, which was equally supported by CFRS. (3)

Considering the nexus between science assessment and research integrity as a major emerging issue that demands attention from the science community, CFRS is further developing the discussion by elevating the matter to the systemic level. With this perspective, it already consented with the “San Francisco Declaration on Research Assessment” (4) that was adopted by a group editors and publishers of scientific journals in December 2012 and critically discusses the use of journal impact factors to measure scientific output to reward scientists and research institutions. In a recent initiative, CFRS jointly with the China Association for Science and Technology (CAST) and the Chinese Academy of Sciences (CAS) sponsored an international workshop in Beijing in April 2014, examining the links between science assessment and research integrity by consideration of the examples of the rapidly developing science systems in Brazil, China, and South Africa. (5)

Science Assessment and Research Integrity

Fundamental to high quality science is the promotion of excellence through quality standards that include peer review of scholarly works and data as well as awards to recognize the highest achievements. As the science enterprise is growing and becoming increasingly competitive, additional measures to assess scientific output were developed by other stakeholders, e.g. journal publishers and science managers. These are mostly quantitative, notably metrics such as the citation and impact factors related to published research applied to individuals, journals, and institutions as well as ranking systems applied to institutions or countries.

The International Council for Science is concerned that the increasing weight of quantitative assessment over peer review could create a reward system that favours quantity of output over quality of output and thereby undermines scientific integrity by tempting scientists to use “shortcuts” to achieve high quantity output. Informed by the discussions at the workshop in Beijing mentioned above, the Council and its CFRS Committee make the following proposals to ameliorate the situation.

Issues of Concern and Proposals for Amelioration

· Purpose and evaluation parameters of assessments

Assessment methods and metrics are applied in different contexts, at the level of countries, institutions, disciplines, and individual scientists, while assessment results are used for different purposes, such as allocation of research funds, promotion of scholars or goal setting. Science assessments may be helpful management tools, but it is critical that their purpose be defined transparently.

· Align assessment output with science values and incentives

Quantitativeassessments are generally based on easily measured dimensions such as output or citation volume. There needs to be attention given to measurement of less readily quantified factors such as impact on the discipline or relevance to society.

· Science and science management needs

Many quantitative assessment systems were developed to facilitate the efficient allocation of scarce resources, rather than to reward merit. The science community needs to be involved more strongly in assessment systems, to ensure that there is concurrence between quantitative evaluations and evaluations based on rigorous and unbiased peer review. Assessment systems themselves should be periodically evaluated.

· Ranking systems for benchmarking only

Although ranking systems can provide valuable information on some dimensions of the scientific enterprise, they are inherently limited in capturing the full dimensions of science quality. Their use should be limited to benchmarking and goal setting.

· Reward systems to incentivize research integrity

All assessment systems provide incentives for behavior. Quantitative metrics that focus on impact and citation factors may increase the pressure to rapidly produce scientific output. These conditions provide potential incentives to breach research integrity, for example in the form of multiple publications of the same data, fabricated, falsified or plagiarized data, honorary or even fraudulent authorship, all of which harm the scientific enterprise. The science community and science managers together need to discuss systemic measures to establish incentives that prevent such practices.

· Education in research integrity

An understanding of the principles of research integrity and the behaviour that constitutes improper scientific conduct need to be taught and nourished during every phase of a scientist’s education. Although there are socio-cultural differences between regions in the role of authority, originality, and responsibility within the scientific endeavour, these areas need to be addressed with consistent teaching of principles and codes of conduct at higher education institutions around the world. Violations of research integrity need to be addressed within a preventative and educational framework.

Continued Engagement

The International Council for Science will strive to engage its membership to ensure that research evaluation includes rigorous peer review and to support production incentives that foster research integrity. CFRS will continue its work in that direction, including by contributing to the 4^th World Conference on Research Integrity in Brazil in 2015 (see conference announcement, page 31).

www.wcri2015.org

Dr. Roger Pfister <roger.pfister@icsu.org> is Executive Secretary of ICSU Committee on Freedom and Responsibility in the conduct of Science (CFRS) and Head of International Cooperation at the Swiss Academy of Sciences, which hosts the CFRS Secretariat (2010-2015).