Before defining performance criteria we must agree on what a “qualitative test procedure” is

What is a qualitative test?

Rapid tests of various types such as screening tests for drugs of abuse or for Strep A antigen are numerously performed laboratory investigations. Sometimes these types of tests are called semi-quantitative and sometimes qualitative. Both terms are ambiguous. The International Union of Pure and Applied Chemistry (IUPAC) defines qualitative analysis as tests for classification of nominal properties; “Analysis in which substances are identified or classified on the basis of their chemical or physical properties, such as …” [1]. This differs from the Clinical and Laboratory Standards Institute (CLSI) use of qualitative tests for “… methods that provide only categorical responses (i.e., positive/negative or yes/no)” [2].

Another way to categorize these types of examinations is according to the results, which can be either nominal (a result of a classification) or ordinal (a grading). Nominal and ordinal values can be very accurately defined, and are thus not by definition semi-quantitative in the sense that the results have a large uncertainty. Therefore, the use of the term semi-quantitative is not recommended. Nominal and ordinal values are not restricted to binary scale examinations. A urine test strip for albumin is an example of a “multinary” ordinal examination procedure with a possible outcome of more than two alternative values.

Results for an ordinal quantity are measurement results [3], although the results have no measurement units. Ordinal values are ordered in some way by magnitude. This excludes, e.g., alphabetical ordering of the values. The values “yes” and “no”, included in the CLSI use above, cannot be ordered by magnitude, and are therefore not ordinal values. The results “positive” and “negative” are usually regarded as ordered by magnitude (or 0/1), in the sense that positive implies more of something, e.g., a concentration above a certain cut-off limit. A pregnancy test might be either negative or positive, depending on the concentration of hCG in a urine sample. The results “negative” and “positive” can be interpreted as ordinal values, e.g., <5 mIU/L and >5 mIU/L, respectively.

Results with the binary outcome yes/no are often nominal. Results from a PCR test that detects a specified mutation, e.g., for BCR-ABL1 gene can be reported with “yes” (with the interpretation “…gene is present”) or “no” (with the meaning the “…gene is not present”), without any respect to a possible concentration of the gene. In this case the outcome of the test is not an ordinal measurement but a classification, or a nominal examination [4]. One characteristic of these types of nominal examination results, e.g., for the appearance of gene or classification of a disease, is that values have no individual biological variation. The results are discrete and a true value in between “yes” and “no” is not possible.

Urine test strips is a classic example of “multinary” ordinal test. The possible test values are categorized in multiple arbitrary scale steps with result classes without measurement units [5], although the name of the scale steps may mimic measurement units.

Measurement characteristics of ordinal tests

The concepts of precision, trueness, uncertainty and traceability for ordinal and nominal examination results are not clearly defined. The counterpart to the metrological terminology and description of measurement traceability and uncertainty of results for ratio scale results that is given in ISO 17511:2003 [6] remains to be developed for nominal and ordinal results. Uncertainties for ordinal and nominal examinations might be expressed as probabilities of correct or false results, which are concepts that can be understood for a set of results, e.g., 1000 samples, but is more difficult to understand in relation to a single positive or negative result.

For ordinal examination results with a quantitative, often ratio type, background scale we also lack a clear definition of the detection limit under which most of the results are expected to be negative, and an upper concentration limit above which most test results are expected to be positive. In the interval between these two limits either negative or positive results can be expected by chance. This interval might be termed the gray zone, unreliability region [7], or the “C₅–C₉₅ interval” as suggested by CLSI.

C₅ describes the concentration below which <5% of the results are positive, C₉₅ is the concentration above which >95% results are positive when many replicates are measured.

The C₅₀-value, the equivalence point or the cut-off level, is the concentration where half of the test results become positive and half negative. One method to estimate the equivalence point has been developed through rankit transformation [8] of the accumulated frequency of positive results in relation the logarithmic concentration. The same principles can also be applied to calculate C₅–C₉₅ interval. The C₅₀-value has the benefit, compared to the C₅- and C₉₅-values, that it can be estimated with higher precision. For statistical reasons a much larger number of measurements is needed in order to estimate C₅ and C₉₅ with certainty.

When the C₅–C₉₅ interval is narrow, the ordinal test is sometime called more precise. For tests with multiple scale steps a C₅–C₉₅ interval can be described for each of the steps. For a typical urine test strip the C₅–C₉₅ interval for each step covers the range of concentration corresponding to several scale steps [5]. For multiple scale steps to be meaningful the test must be precise enough with narrow C₅–C₉₅ interval.

As mentioned above traceability and other metrological terms for nominal and ordinal results have not yet been clearly defined. For ordinal results the relation between the C₅₀-value and a reference method value seems to be an important part of the description of the traceability for the ordinal results. Ordinal measurement trueness can be described as closeness of agreement between the equivalence point for a binary ordinal test and a reference quantity value.

Analytical performance goals for ordinal tests

Internal quality assessment (IQA) consists of samples for which the expected results are known and predictable. For ordinal tests the internal quality control therefore must consist of samples with concentrations outside the C₅–C₉₅ interval. Samples with concentrations within this interval will give results that are unpredictable by definition. Therefore these are not useful as internal quality control samples, for the purpose of a check that the measurement procedure is under control.

For samples used in external quality assessment (EQA) the values must be unknown for the participants before examination. Performance for samples with concentrations outside the C₅–C₉₅ interval is a measure of performance of the individual laboratory or participant. However, contrary to the case with IQA, results for samples within the C₅–C₉₅ interval are also informative in the context of EQA. The outcome of these allows the estimation of the C₅₀ value for a specific test procedure, which is often an in vitro diagnostic product on the market. In this way EQA for an ordinal test becomes a measure of ordinal trueness for the procedure.

In addition to estimation of trueness, the robustness and selectivity must be challenged in EQA. The analytical specificity should be checked either by using native sample materials, with natural occurrences of molecular species and antigens other than the target analyte, for which it is important that the test is specific.

Analytical performance goals for ordinal scale test procedures must be set according to the intended use of the procedure. When the test is used for a screening purpose, and a positive test will be confirmed with a more specific secondary test, then the performance goal for the screening procedure can be a very high sensitivity. In the case of a screening test for drugs of abuse, the cut-off limit and other performance criteria might be set for legal reasons.

In other situations an ordinal scale test is primarily used to exclude a positive result. This might be the case for a Strep A antigen rapid detection test, that is used with the purpose to avoid antibiotic treatment for patients with sore throat, but without group A Streptococcus antigen. In this situation a high specificity is preferred to a high sensitivity. A negative result should be reliable.

It is usually not possible to base performance goal for ordinal test on biological variation. The gray zone, or the C₅–C₉₅ interval, is probably larger than the biological variation for many measurands.

The performance goals have so far probably been based mainly on state-of-the-art for ordinal scale tests, e.g., urine test strips. With the future technological development the performance of many ordinal scale tests can be expected to improve.

Another area in need of standardization is the mode of reporting ordinal test results. Results for test strips are today reported in several different ways: positive/negative, or 0/1, –/+/++, “approx. 10–20” etc. These multiple ways to report the same result is confusing and leads to risk for misinterpretation when results are registered in health records and communicated.

Analytical performance goals for nominal tests

Performance goals for nominal examination procedures must be set with respect to the type of test. For genetic tests and, e.g., blood group determination, a negligible minimal examination uncertainty is essential for the proper use of the results. For other types of nominal examinations, e.g., cell classification based on morphologic appearance, the uncertainty might be considerable.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Financial support: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.