Harmonization of microbiology processes and standards: work in progress

Introduction

Clinical microbiology laboratories incorporate a wide range of methods and technologies into a single discipline. These range from traditional methods such as the Gram stain, which originated over 130 years ago, to novel technologies such as next-generation sequencing. Many of these processes are not only labor intensive but also technically challenging and require interpretation of complex test results by the microbiologist. Traditional methods such as culture are complicated by the need to distinguish normal bacterial flora from pathogens, many of which are found routinely as part of the normal host flora. The clinical microbiologist is expected to make a determination based on laboratory protocols distinguishing pathogen from non-pathogen in a setting where the same microorganism can occupy either niche depending on the context in which it is found. These challenges are further compounded by patient populations that are increasingly immunocompromised, which further blurs the line between host bacterial flora and pathogens. The laboratories in which this testing occurs may vary significantly in complexity – from community hospital-based laboratories with a limited test menu to laboratories serving tertiary care centers with populations requiring specialized testing protocols such as transplant or cystic fibrosis patients.

The current role of metrics in microbiology

A number of groups have published guidelines for the processing and interpretation of cultures [1], [2]. The American Society of Microbiology recently initiated the Evidence-Based Laboratory Medicine Practice Guidelines (EBLMPG) committee in an effort to develop standardized practice guidelines [3]. The goal of this effort was to evaluate existing peer-reviewed literature to develop consensus data-based standards for clinical microbiology laboratories. The initiative has published two guidelines but has in a number of scenarios been unable to provide definitive recommendations due to the lack of data and/or properly structured studies [4]. Accrediting agencies such the College of American Pathology (CAP) have attempted to provide guidance via their laboratory accreditation and proficiency programs [5]. Proficiency testing regimens are required by laboratory accrediting bodies, but the absence of standardized reference materials that truly mimic the complexity of patient samples makes it challenging to maintain or measure interlaboratory consistency. The harmonization of microbiology protocols is made all the more difficult by manual processes that vary based on specimen and patient type. Cultures for aerobic and anaerobic bacteria differ significantly from cultures for mycobacteria and fungi in terms of sample processing steps, media used and incubation times. Sputum samples for culture may be processed and results interpreted differently depending on whether the patient has cystic fibrosis, bronchiectasis, legionellosis, nocardiosis or community-acquired pneumonia. Urine cultures may utilize different thresholds for clinical significance depending on the method of specimen collection. The wide range of protocols and pathways can lead to protocol drift over time and variation in processes within laboratories with staff spread across multiple shifts that can be difficult to monitor.

In order to address some of these concerns, accrediting agencies and standards such the CAP and ISO 15189 require that laboratories have a process for detection of significant errors [5], [6]. This may include secondary review of culture reports or use of electronic flags in laboratory information systems [5]. The absence of specific guidance on the standardized metrics to be utilized or the definition of significant errors makes it challenging to generate meaningful data that can be compared across diverse laboratories. The process may have limited utility because it does not provide oversight of the actual testing performed by the individual microbiologist and does not easily lend itself to harmonization. The consolidation of clinical microbiology laboratories into high volume testing facilities makes it even more challenging to perform routine review of all test processes even when utilizing electronic tools.

In this setting, metrics have become an increasingly essential means of monitoring and maintaining quality. The 15189:2012 laboratory standards emphasize the need for tracking errors and addressing root cause [6]. It states “the laboratory shall evaluate the impact of work processes and potential failures on examination results as they affect patient safety, and shall modify processes to reduce or eliminate the identified risks and document decisions and actions taken” [6]. Laboratories often tend to focus on the analytical aspect of the testing process, but studies have shown that these represent only 15% of all errors during the total testing process (TTP) [7], [8]. The ISO 15189:2012 standards require that laboratories monitor incidence of errors in all stages of the testing process from pre- to postanalytical.

The use of metrics requires [9]

1. standardization of quality indicators between laboratories;

2. uniform data collection methods.

The field of clinical chemistry has made coordinated efforts to address both requirements via the International Federation of Clinical Chemistry and Laboratory Medicine [9]. Although metrics are widely used in clinical microbiology, the lack consensus on common metrics, standardized collection methods and the inability to compare data between laboratories have severely limited their utility in harmonization of the TTP. There is also significant variation in protocols between laboratories, which is further exacerbated by the needs of diverse patient populations. This lack of consistency means that laboratories cannot compare their performance against established standards and struggle to use metrics in an effective manner. There is also a dearth of studies showing interlaboratory comparison of performance for routine culture methods. Applying lean principles to this issue, the problem cannot be addressed without first determining the extent of variation between laboratories. Metrics can be a part of that solution, but only if applied in a standardized manner.

Studies evaluating quality metrics in clinical microbiology have primarily focused on error rates but are often limited to single institutions. A number of publications have examined error rates in blood culture Gram stains [10], [11], [12]. Goodyear et al. examined errors along the TTP for positive cultures and determined their error rate to be 0.8% of which 34% were clinically significant [13]. Yuan et al. assessed the clinical impact of laboratory errors, which resulted in corrected reports, and found that 6.7% of corrected reports resulted in adverse clinical impact [14]. Multicenter studies examining error rates in clinical microbiology have been relatively limited in scope or used surrogate markers for quality in part due to the challenges of standardizing data collection processes. Morris et al. and Church et al. used laboratory proficiency performance to evaluate performance of laboratories across multiple sites, whereas Goodyear et al. used a computer-based competency assessment tool to examine Gram stain performance [15], [16], [17]. Although these studies provide valuable information on the use of quality indicators to measure laboratory performance, the lack of standardized definition and classification scheme for errors makes it challenging for peer laboratories to use these data effectively. The first multicenter assessment of laboratory errors using a standardized approach for both definition of errors and data collection/analysis in clinical microbiology was performed by Samuel et al. on Gram stains from specimens obtained from sterile sources [18]. They determined that the Gram stain error rate ranged from 0.4% to 2.7% between sites, but the proportion of errors that were due to technologist error significantly varied between sites (9%–45%) [18]. This further emphasizes the diverse issues that impact clinical microbiology laboratories and the need for collaborative efforts for the harmonization of both protocols and quality metrics.

Metrics as tool for harmonization

A wide variety of metrics have been utilized by clinical microbiology laboratories. Some have been mandated by regulatory bodies such as the CAP, whereas others are generated by laboratories to address gaps in quality assurance systems and proficiency programs. There are few consensus metrics, nor are there guidelines that serve to define metrics, their collection process or their acceptable limits. As a consequence, laboratory information systems are often poorly equipped to handle the need for data collection. Even in cases where the metrics are required by regulatory bodies such as CAP, which requires laboratories to routinely monitor blood culture contamination rates and intervene when contamination rates exceed certain thresholds, not all laboratories utilize the same definition for a contaminated blood culture [5]. Other metrics during the preanalytic phase also required by the CAP include blood culture fill volume [5]. A survey of the ASM listserve for laboratory directors (Clinmicronet) revealed some of the metrics that clinical microbiology laboratories track (Dr. A. Harrington, personal communication) (Table 1). The laboratories using these metrics face a number of challenges in implementing their findings in an effective manner.

Sputum culture contamination rate: specimens for sputum culture are prone to contamination by oral flora. CAP requires that laboratories screen specimens for acceptability using the Gram stain [5]. CAP, however, does not define the screening process, and laboratories take widely varying approaches in setting the standards for what constitutes an unacceptable sputum sample [19], [20]. In addition, there are no guidelines that advise laboratories on what could be considered an unacceptable rate of contamination that requires retraining of staff in optimal specimen collection practices.

Urine culture contamination rate: improper specimen collection practices can lead to contaminated urine cultures. Laboratories may choose to track urine culture contamination rates although there is little agreement as to what would be considered an unacceptable contamination rate. The EBLMPG guideline on preanalytic steps for urine culture was unable to make specific recommendations for a number of aspects of the specimen collection process because of the lack of systematic studies [4].

Acid fast bacilli (AFB) culture contamination rate: culture of AFB from specimens such as sputum that contain significant quantities of normal flora can be challenging because the prolonged incubation time (6–8 weeks) and slow growth rate of AFB lead to loss of cultures due to contamination by faster growing bacteria. Labor-intensive manual decontamination processes are required to reduce contamination rates, but these can lead to loss of AFB if not properly controlled [21]. Tracking culture contamination rates is an essential aspect of mycobacterial culture quality assurance. Inadequate decontamination can lead to high rates of contamination, whereas conversely, overly harsh decontamination can lead to false-negative cultures. According to the World Health Organization laboratory guidelines, a contamination rate of 2%–3% is considered acceptable for fresh sputum specimens, whereas other sources have suggested that a contamination rate of 3%–5% is recommended with the Association of Public Health Laboratories specifying a range of 2%–5% for solid culture media and 7%–8% for liquid culture media [21], [22].

Interferon-γ release assay (IGRA) indeterminate rate: IGRA assays are prone to indeterminate results due to inappropriate collection or immunocompromised patients [23]. Tracking indeterminate rates may allow laboratories to identify individuals that need retraining. Indeterminate rates may vary based on patient population, and there are no guidelines as to the expected rate of indeterminate IGRA results due to non-collection-related issues.

Specimen processing time: many laboratories track time required for processing specimens. In particular, specimens with high clinical impact such as blood cultures. Consolidated laboratories often receive specimens from a network of hospitals via a courier system. Prolonged transport time can have a detrimental effect on specimen quality/organism survival and potentially impact patient care negatively. These metrics include not only time required for transport and processing of blood cultures on receipt but also reporting of positive blood cultures as critical values. Accrediting bodies recommend timely processing of positive blood cultures but do not define specific time frames, leaving it open to interpretation. This can be particularly challenging for smaller community hospitals where the microbiology laboratory is not staffed 24/7 or staffed primarily by generalists with shared duties. The absence of consensus requirements for specimen processing makes it difficult for laboratories to justify the resources required to address these concerns.

Gram stain/culture correlation: correlation of Gram stain and culture results can give insight into the appropriate test selection, culture practices and Gram stain performance. Data from review of ~42,000 culture results at a large clinical microbiology laboratory (Dr. R. Cavagnolo, personal communication) indicated 94% correlation between culture and Gram stain for non-blood specimens with positive cultures showing moderate/many colonies in routine aerobic cultures. Using this subset of specimens, Samuel et al. examined the incidence of errors in Gram stains from sterile specimens [18]. Although the percentage of specimens with discrepant Gram stain/culture results was relatively consistent across all four study sites (4%–6%), the incidence of errors among these discrepant results ranged from 9% to 45% [18]. This again suggests significant variation in performance between laboratories and the need to develop these metrics to harmonize standards of care.

Corrected reports: few studies have examined the incidence of corrected reports in clinical microbiology [13]. These can and do have a significant clinical impact and can serve as a valuable marker for quality in the clinical microbiology laboratory [14]. However, there are no standard definitions of errors that require corrected reports, nor have been these been categorized by type of error and clinical impact. Harrington et al. examined incidence of technical and clerical errors in cultures from sterile specimens across four tertiary care centers and found the error rates ranged from 4% to 19% per site [24]. However, there was significant variation in the data collection methods between sites that could have contributed to variation in error rates. The average error rate was 10% with 69% being clerical errors and 31% being technical errors. Approximately 25% of the errors had potential clinical impact [24]. These data suggest that there is significant room for improvement in performance in clinical microbiology laboratories and that harmonization and broad application of these quality metrics is essential in making progress in this arena.

Positivity rate for molecular assays: as clinical microbiology laboratories transition to molecular-based assays over traditional culture methods, the need for performance standards becomes even more acute. Failure to adhere to strict protocols can lead to contamination events and cause patient harm. Laboratories often monitor positivity rates for molecular assays as a means for tracking potential contamination [5]. However, these rates could vary between laboratories based on patient population and the platforms utilized for testing.

Harmonization of testing processes within microbiology is also challenged by the fact that there is often lack of consensus on the optimal protocols for testing. There is significant disagreement on the use of PCR alone rather than algorithm-based approaches for the diagnosis of Clostridium difficile [25]. Guidelines suggest that laboratories may use either approach even if there may be differences in test performance characteristics [26]. The advent of molecular-based technologies has significantly changed the paradigm for microbiology testing but has not necessarily improved the prospects of harmonization. Although multiplex molecular testing offers significant advantages over culture, the lack of data to guide test utilization, test performance and impact on patient care is problematic [27]. The development of international standards for quantitative molecular assays has been a major advancement in the standardization of viral load results, but interlaboratory standardization remains an elusive goal [28].

Conclusions

Laboratories struggle to make evidence-based decisions that balance the needs of the patient with the limited resources available to the laboratory. This contributes further to the lack of harmonization in test performance standards between clinical microbiology laboratories. This in turn has serious implications beyond laboratory quality assurance and brings into question the effectiveness of clinical guidelines that rely on laboratory results. The field of clinical microbiology has a long way to go to develop consistent, comprehensive and widely accepted standards and metrics. Coordinated efforts and strategies are needed to make progress in this area.