International Evaluation Program Reveals True Situation

Quality of Chemical Measurements

International Evaluation Program Reveals True Situation

by Philip D. P. Taylor, Ioannis Papadakis, Lutgart Van Nevel, Ciaran Nicholl, and Prof. Paul De Bièvre*

Introduction

Background

The Problem

The BIPM Gets Involved

IRMM’s International Measurement Evaluation Program (IMEP)

A Practical Example: IMEP-9 Trace Elements in Water

The Way Forward

Introduction

The wide variation in chemical trace measurements was first brought to my attention by Prof. De Bièvre at the 1998 meeting of the Committee on Chemistry and Industry (COCI) in Johannesburg, South Africa. During our 1999 meeting in Berlin, Germany, he again presented data demonstrating a huge (±50%) variation in trace element (Pb, Cd, Fe, and Zn) concentrations in water even from "accredited" laboratories. Similar problems were demonstrated for analysis of catalyst metals (Pt) in car exhaust material. Because such data are increasingly used in decision-making areas of industrial concern, our committee adopted "Reliability of Chemical Measurements" as a new, formal project, with Prof. De Bièvre as coordinator. Interest was immediately demonstrated by UNESCO (Dr. A. Pokrovsky), and cooperative information dissemination efforts began. Although a brief summary of the concerns, "Metrology in Chemistry", had been published in Chemistry and Engineering News (C&EN, 31 May 1999, p. 29), COCI encouraged Prof. De Bièvre to submit a more detailed summary to Chemistry International (CI). This article provides "snapshot" pictures of chemical measurement (un)reliability, with many practical, societal implications.

Dr. A. Nelson Wright

Chairman, IUPAC Committee on Chemistry and Industry (COCI)

Background

Chemical measurements form the basis of many important economic, political, environmental, medical, and legal decisions. Each day, millions of such measurements are carried out throughout the world. The real basis for decision-making and implementing regulations depends on the comparability and reliability of the results of these chemical measurements. The European Commission requires that measurements performed in one Member State must be acceptable to all other Member States in the Union. This process requires that such reliability be demonstrated. Moreover, the results of European chemical measurements must also be transparent and clearly understood by Europe’s trading partners in the whole world and vice versa. Globalization of commerce and the need for fair trade require knowledge of the degree of equivalence of the measurement results as they affect the value of traded goods.

* IUPAC’s Committee on Chemistry and Industry (COCI) has submitted the above article, prepared by Dr. Philip D. P. Taylor (Joint Research Centre-European Commission, Institute for Reference Materials and Measurements, Retiesweg, B-2440 Geel, Belgium; E-mail: taylor@irmm.jrc.be), Dr. Ioannis Papadakis (papadakis@irmm.jrc.be), Mrs. Lutgart Van Nevel (vnevel@irmm.jrc.be), Dr. Ciaran Nicholl (nicholl@irmm.jrc.be), and Prof. Paul De Bièvre (Duineneind 9, B-2460 Kasterlee, Belgium; E-mail: paul.de.bievre@skynet.be). COCI Chairman Dr. A. Nelson Wright (12539 Ranger, Montreal, Quebec H4J 2L7, Canada; E-mail: anwright@citenet.net) contributed the Introduction.

Quality of Chemical Measurements (cont')

International Evaluation Program Reveals True Situation

Introduction

Background

The Problem

The BIPM Gets Involved

IRMM’s International Measurement Evaluation Program (IMEP)

A Practical Example: IMEP-9 Trace Elements in Water

The Way Forward

The Problem

The comparability of chemical measurement results demands, where possible, that they should be traceable to "stated references" (i.e., values) and preferably expressed in values of the International System of Units (SI–Système International d'Unités). Traceability to such values ensures comparability (results can be validly compared).

Many modern analytical methods are based on comparative techniques that rely on measuring the ratio of an instrumental signal for an unknown sample with that of a known "standard". Very few chemical measurements are being performed with a clearly stated traceability of their results. Accreditation schemes can improve the quality of measurements by making them comply with formal procedures, but the problem of the reliability and degree of equivalence of the results must be addressed, and this issue has been tackled only recently. The essence of the problem lies in applying the principles of Metrology in Chemistry (MiC). Metrology is still rather new in chemistry but will grow according to the increasing emphasis on establishing the degree of equivalence and the need for demonstrating the true reliability of the measurement results.

The BIPM Gets Involved

A selection of certified isotopic reference materials

The International Consultative Committee on Amount of Substance (CCQM–Comité Consultatif pour la Quantité de Matière) was set up in 1995 by the Paris-based International Bureau of Weights and Measures (BIPM–Bureau International des Poids et Mesures). Key tools of MiC are primary methods of measurement, defined by the CCQM as methods for which a complete uncertainty statement can be written down in terms of SI units and whose results are accepted without reference to a standard of the quantity being measured. So far, the CCQM recognizes five such methods, namely, titrimetry, coulometry, gravimetry, determination of freezing point depression, and–most versatile and widely used–isotope dilution mass spectrometry (IDMS), which has been applied and refined at the Institute for Reference Materials and Measurements (IRMM) for over 30 years.

The basis of IDMS is that one measures an unknown number of atoms of an element (or molecules of a compound) in a sample through direct comparison to a known number of atoms of the same element (or molecules of the same compound) in an amount of a spike added.

IRMM’s International Measurement Evaluation Program (IMEP)

IMEP-10 bottle for the determination of Cd, Cr, Hg, Pb, As, Cl, Br, and S in polyethylene

In 1988, IRMM founded the International Measurement Evaluation Program (IMEP) as an awareness program and as a tool to show the true state of chemical measurement results, both to practitioners and to end-users of chemical measurements. The IRMM is part of the European Commission’s Joint Research Centre, which is made up of eight research institutes spread over five sites. The mission of the IRMM is to promote a common European measurement system supporting the implementation and monitoring of Community policies, by

• developing and performing specific reference measurements,

• producing Certified Reference Materials (CRMs),

• organizing International Measurement Evaluation Programs,

• establishing community databases,

• performing prenormative research related to European norms or standards, and

• offering special high-level training in measurement sciences.

IRMM has developed its expertise and facilities to be able to deliver the highest quality of primary measurements. Host to the largest isotope mass spectrometry facility in Europe, IRMM also has extensive technological material preparation facilities. This unique combination yields Certified Test Samples (CTS) in "real life" matrices fully characterized for homogeneity and stability.

IMEP runs in adherence to the IRMM mission. Today IMEP also runs under the auspices of IUPAC, Analytical Chemistry in Europe (EURACHEM), European Organization of Metrology (EUROMET), and Cooperation on International Traceability in Analytical Chemistry (CITAC).

IMEP enables field laboratories to compare their results against SI-traceable reference values. CTS with undisclosed values are sent to interested participants. The participants are asked to return values together with uncertainty statements claiming in writing to contain so-called "true" values. The undisclosed reference values are established by measurement procedures based (where possible) on IDMS applied as a primary method of measurement. Thus, IMEP basically demonstrates the degree of equivalence of results of chemical measurements for individual laboratories on the international scene as evaluated against results of primary methods of measurement.

• Laboratories evaluate themselves under normal conditions, and they may choose their own techniques, procedures, and instrumentation (preferably routine conditions). They are requested to report their results with a best estimate of combined uncertainty according to ISO/BIPM guidelines.

• IMEP is open to all laboratories, and full confidentiality with respect to results and participant’s identity is guaranteed.

• IMEP focuses on "real life" samples that represent highly important areas (food, environment, health, etc.).

• IMEP graphically displays reference values and results from participating laboratories, but the conclusions are to be drawn by the laboratories themselves. Additional support can be offered by IRMM after the measurement round.

• IMEP is coordinated by IRMM. The establishment of reference values with their uncertainty (which gives a reference range) is a timeconsuming and expensive task. It is accomplished in collaboration with several partners–worldwide "IMEP reference laboratories" that can demonstrate a track record of successful applications of IDMS as a primary method of measurement.

• IMEP is complementary to collaborative studies and proficiency testing schemes; the latter are more frequent, but have a weaker metrological basis. Since 1988, IMEP rounds have mainly concentrated on trace elements in various matrices such as water, polyethylene, serum, and sediment. Thus far, 13 IMEP rounds have been completed, as described in Table 1. At present, IMEP rounds 12, 16, 17, 18, and 19 are ongoing, as listed in Table 2. The first IMEP round, carried out over 10 years ago, addressed the issue of measuring lithium in human serum. Six analytical laboratories at prominent psychiatric institutes were asked to participate. Lithium is administered to patients suffering from mental disorders, and the alarming fact was that all laboratories reported incorrect measurements that were off by a factor of up to five in some cases. One laboratory reported a therapeutic concentration that was actually toxic.

Table 1 Completed IMEP rounds.

Table 2 Ongoing IMEP rounds.

[More]

Quality of Chemical Measurements (cont')

International Evaluation Program Reveals True Situation

Introduction

Background

The Problem

The BIPM Gets Involved

IRMM’s International Measurement Evaluation Program (IMEP)

A Practical Example: IMEP-9 Trace Elements in Water

The Way Forward

A Practical Example: IMEP-9 Trace Elements in Water

The IMEP rounds related to water analysis (3, 6, and 9) traditionally have had a large number of participants, thus reflecting people’s interest in water analysis. This interest is also expressed through several EU directives and national legislation that specifically addresses this topic.

Typically, each laboratory is requested to declare its level of competence as accredited/non-accredited and/ or self-declared experienced/less experienced to enable further interpretation of the data sets.

In all, three water IMEP rounds–and for most elements about 90% of the laboratories–reported values within 50% of the certified value. In a small numnber of cases, the reported values deviated from the certified value by more than 50%. Undoubtedly, these results should be judged by taking into account the required regulatory quality of the measurement (e.g., ±10% in EC Directive 98/83/EC). The data (see Fig. 2) show that for most laboratories, this requirement poses a problem.

In IMEP-9, several "regional coordinators" acted on behalf of IRMM to liaise with participants, bridge linguistic and cultural differences, and take local particularities into account.

Yetunde Aregbe (Austria) performing the highest quality isotopic measurements with IRMM's Avogadro II Mass Spectrometer, which is the world's first "amount comparator".

These coordinators were either people or institutions directly involved with chemical measurements, having a certain degree of experience and competence in metrology, and with links in the measurement systems of their country or region. Almost 85% of the participants were handled by the 13 regional coordinators, as shown in Table 3. Results were reported by 201 participants from 35 countries and 4 continents. A distribution of the participants as a function of the country is shown in Table 4.

IMEP in Support of a Global Metrological Structure for Chemical Measurements Samples used in IMEP-9 were offered to CCQM to be used in a key comparison (CCQM-K2), and nine National Measurement Institutes (NMIs) from all over the world measured these samples. Thus, the field laboratories (IMEP-9 participants) were able to compare their results with NMI results and vice versa. This exercise was one of the first-ever BIPM key comparisons in the area of chemical measurements, and it was successfully completed with IRMM as the pilot laboratory.

Fig. 1 Results from IMEP-9 NMIs >Click to enlarge	Fig. 2 Results from all laboratories >

The importance of applying metrology to chemical measurements is clearly demonstrated by comparing the results obtained by the NMIs (Fig. 1) to the results obtained by the field laboratories (Fig. 2). The certified value for lead in the IMEP-9 CTS was 62.3 ± 1.3 nmol/L. Eighteen of the 181 participating laboratories that measured Pb reported values more than 50% above the certified value, while 4 laboratories reported values 50% below. On the other hand, the NMIs participating in CCQM-K2–using a PMM test instrument–reported values within a fraction of a percent of each other and of the reference value.

Table 3 Regional coordinators of IMEP-9*

*Results were reported by 201 participants from 35 countries and 4 continents.

Table 4 Participants' country of origin

The Way Forward

IMEP illustrates the need for a structured measurement system for chemical measurements. In such a system, various organizations and laboratories need to take on their responsibility, e.g., by agreeing on who assures demonstrated measurement capability for a particular measurement. IMEP is unique because it enables one to view the claimed measurement capability of different laboratories at various metrological levels, with an international perspective.

IMEP will continue to foster this perspective and grow, with particular focus on laboratories in the Member States of the EU as well as in the EU preaccession countries. In order to prepare the latter for integration into the EU’s measurement and accrediation systems, IRMM is offering:

• postdoctorate fellowships (2–3 years)

• detached national experts

• visiting scientists

Ellen Poulsen (Denmark) preparing graphs for the IMEP-9 participants; report on trace elements in water.

In this way, IRMM will give training/advice on traceability and on uncertainty evaluation of chemical measurement results. Participants can take this expertise back to their home countries and/or become regional coordinators for IMEP in their respective countries. In collaboration with EUROMET and CCQM, IMEP samples are offered for the organization of EUROMET key or supplementary comparisons and, where appropriate, for the organization of CCQM key comparisons or pilot studies. In its role as a neutral, impartial international evaluation program, IMEP displays existing problems in chemical measurement. IRMM is dedicated to tackling this problem and will, where possible, collaborate with international bodies, education and accreditation authorities, and NMIs to achieve more reliable measurements and contribute to setting up an internationally structured measurement system.