On good reporting practices for property measurements

Have you ever said, “Eureka! This citation should contain precisely the experimental data I need to quantitatively prove my theory and to design the novel process it enables.”? But then, when you examine the full article, you find that no numbers are given, only graphs. Or maybe there are tables of values, but the units are not specified. Perhaps the data are not original, but their source is not given. Closer inspection might reveal that derived property values of little interest are tabulated, but no primary data to get properties of your interest are provided. Your frustration grows when you recognize that the experiments were probably done carefully, but the reporting of the results leaves so many omissions and uncertainties that you cannot confidently use the results.

If this has been your experience, you might relate to the scenario of the video, “Data Sharing and Management Snafu in 3 Short Acts” from the NYU Health Sciences Library [1].

Unfortunately, these situations occur far too often in the scientific literature across all disciplines. A huge amount of valuable information has been, and continues to be, lost forever because their publications were incomplete or imprecise. We have gathered a large number of real publications with inadequate data reporting of experiments on thermophysical and thermochemical properties. Here are a few representative examples:

Reader: “What I need is the data. Can I have a copy of your data?” Author: “Everything you need to know is in the article.” Reader: “No… I think I cannot use your data”. Reproduced from [1] by permission from Karen L. Yacobucci (NYU Health Sciences Library)

1. Researcher A had a research grant to find a solvent for a specific practical application. As part of the research, many solubilities were measured, but no values were published. Rather, only a discussion of the relative suitability of different chemical classes of potential solvents was presented, so no applications could be done.

2. Researcher B studied heat capacities of novel substances with precise adiabatic calorimetry having a relative uncertainty of ± 0.1 %. However, all that was published were plots of poor resolution over a wide range of temperature. Digitization could only yield degraded data, so the extra effort for precision was wasted.

3. Researcher C measured vapor pressures for several organic compounds yet published only derived enthalpies of vaporization and Antoine equation parameters with no units or the valid temperature range. Moreover, some correlation parameters were misprinted, preventing any reliable use of the work.

4. Researcher D measured and published enthalpies of dilution of electrolyte solutions but did not indicate whether they were based on moles of solute, of added solvent, or of something else. All attempts to interpret those data in a meaningful way failed, and research that was potentially valuable for modeling electrolyte solutions was wasted.

5. Researcher E studied a mixture of water with a partially miscible deep eutectic solvent (DES) made from compounds A and B in a molar ratio of 1:2. The listed mole fractions of DES in the aqueous solutions did not give a basis for the results (1 mole of {A +2B} or of {1/3A + 2/3B}), resulting in a factor of three ambiguity in the reported compositions. Also, the report did not address the fact that the components of the DES could be nonuniformly distributed between co-existing phases.

6. Researcher F published solubilities of industrially important gases in water and organic solvents but did not mention the origin and nature of those values: measured, predicted, or taken from another document. This issue has also been found in reports of thermodynamic properties of explosives where enthalpies of formation, combustion, decomposition are predominantly published without any traceability.

7. Expert X has a full-time assistant who digitizes plots of alloy property data from journal articles for use in model development. If the original researchers had published numerical values, not only would the claimed accuracy be preserved, but also a tremendous amount of time and resources would be saved.

The sadness of these situations is that though the measurements may have been carefully done, if the data communication is flawed, everyone loses. The original workers, and their sponsors, do not get appropriate credit, while users waste time attempting to uncover needed information that may not actually be in the publication. The result can even be needless replication that is expensive in labor and equipment. Further, as technology advances, data can be useful for purposes beyond the original intention. Thus, chemical engineers can employ them to design chemical processes, build models, assess process safety, etc. But incomplete or ambiguous reporting will hamper such practical advances.

The existence of the problem was acknowledged some years ago [2], and specific recommendations were developed [3]. But the efforts did not significantly change the situation. That is why a large group of data users and experts in experimental methods joined an IUPAC project to develop the Good Reporting Practice Principles for experimental property data [4] to inform all parties involved in the scientific process (funding agencies, publishers, and researchers). Those principles are:

1. Measured property data should be published in a numerical format (at least in a supplement).

2. Published data should be well defined (including system, state, and property).

3. All published data should be traceable to their origin.

4. Observations should be distinguished from interpretation.

5. Auxiliary (calibration) data should be identified and provided.

6. Necessary details of experimental methods or computation procedures should be given.

7. Uncertainty in each measured value should be reported and justified.

8. Importing reported data into analysis software should be easy and straightforward.

9. Complex mathematical equations should be provided in a machine-readable form.

Also recommended was that corrigenda should be immediately submitted, with prominent visibility from the main publication web page as soon as errors are detected.

Justification for each principle is detailed in the IUPAC Technical Report [4] and its Appendix includes specific examples.

The report also recognizes that well-designed research should solve problems rather than create them. Sometimes problems can be revealed but not immediately solved; however, well-designed research should minimize the number of remaining questions. Therefore, Elements of Good Research Practice have also been included in [4] because reporting issues are frequently tied with the design of experiments. Best reporting might require conducting additional experiments before publication. Following these elements can avoid delays and confusion in properties communication. The Good Research Practice elements are in the following areas:

1. Content (planning): For example, measurements should be made of limiting behavior to provide a basis for consistency checks and model development.

2. Methodology (procedure): The accuracy and reliability of the method and apparatus should be confirmed on well-studied test systems over the whole range of conditions of the study. This element also includes material stability, hygroscopicity, and proof of identity of newly synthesized compounds.

3. Validation: Thermodynamic consistency of the measured values and comparisons with previously published results should be checked.

The Technical Report [4] is not a checklist to be followed for each publication of experimental results, though some properties journals have instituted this responsibility for authors (see references [18] and [19] in [4]). Rather, it is intended as advice for consideration when developing publications.

The principles and elements were developed for the thermophysical/thermochemical fields, but equally apply to other areas of science, as in the video cartoon about medical literature [1].

The Task Force team is initiating efforts to publicize the report, to encourage journals that publish data to educate their authors and reviewers about these techniques of good reporting and research practices, and to suggest that funding agencies include the principles in their proposal materials.

The report [4] concludes, “Acceptance of the Principles would assist in more complete transfer of the knowledge acquired in scientific research. In addition, greater efficiency would be made of the funds distributed for that research to benefit the entire community of taxpayers, readers, and users, as well as the researchers themselves.” These are important goals for contemporary science and technology.

The Technical Report [4] was prepared within the framework of IUPAC project (https://iupac.org/project/2019-013-1-100), “Good Reporting Practice for Thermophysical and Thermochemical Property Measurements.”