Informatics External Quality Assurance (IEQA) Down Under: evaluation of a pilot implementation

Introduction

Researchers from the Royal College of Pathologists of Australasia Quality Assurance Programs (RCPAQAP) and Macquarie University recently published a paper describing the importance of pre- and post-laboratory phases which form the diagnostic phases of laboratory medicine testing [1]. The issues, however, have been known of in Australia since at least 1996 [2]. These phases “beyond the lab” constitute a major source of errors that reduce laboratory effectiveness and threaten patient safety [1], [3]. External quality assurance (EQA) ensures that verification is performed on a recurring basis, and that laboratory results conform to expectations for quality required for patient care [4]; however, most Australian laboratories have previously focused narrowly on laboratory processes such as test accuracy and precision [1]. Until recently, it was very difficult to quantify the number and magnitude of errors in the extra-analytical phases due to a lack of formal EQA measures in these areas.

Detection of these errors requires reliable quality indicators during the total testing process (TTP), from the time the laboratory request is determined, until the clinician receives the final report, makes a diagnosis and decides on the appropriate action [3], [5], [6]. Launched in 2008, the Key Incident Monitoring and Management System (KIMMS) is an Australasian-developed quality improvement (QI) program that records incidents (process defects) and episodes (occasions where incidents may occur) while also assigning quantified risk to each incident type (by multiplying by harm rating and detection difficulty score) using failure mode effects analysis (FMEA) [5]. By 2016 KIMMS had detected over 200 million episodes and 2.9 million incidents, detecting an overall TTP incident rate of 1.75% [5]. Some incident rates may appear low, but when taking into account risks and their frequencies, critical incident types emerge that require improvements in management. For example, haemolysis had both the highest incidence (22.6% of total incidents) and highest risk (26.68% of total risk). However, incidents that have low frequency (e.g. “sample suspected to be from wrong patient” had the second lowest score) but high harm rating (e.g. 10/10) and detection difficulty scores (10/10) end up having a relatively higher risk to reflect the severity of potential risk to the patient [5]. Until recently, KIMMS has focused mainly on the pre-laboratory phase of the TTP cycle, thus there has been a need to create an EQA program that encompasses the post-laboratory phase.

One of the major post-laboratory areas that has been under urgent pressure for improved EQA measures has been laboratory reporting, especially given the widespread adoption of electronic health records which aggregate reports from multiple laboratories, such as MyHealth Record in Australia [7]. The importance for standardisation or “harmonisation” of the formats and styles used in clinical chemistry reporting is key to interoperability and safety for electronic health records [8], [9], [10]. Significant variations in reporting policies between different Australian laboratory medicine providers, or even within the same provider, result in different styles of reports for different customers [11]. The clinical chemistry report needs to be clear and unambiguous; however, in Australia there are still differences in reporting, e.g. different names for the same test, different units, different tests included in panels with the same names, differences in reference intervals (RIs) and flagging of results outside limits [12]. These differences may introduce misunderstanding resulting in interpretation errors by clinicians or patients at the post-laboratory phase [10], [13], which is a patient safety issue. Activities aiming to increase harmonisation in laboratory medicine include improving metrological comparability of results, as well as reducing unnecessary between-laboratory variation in test requesting and reporting [9], which is the focus of our trial.

The Australian Pathology Units and Terminology Standardisation (APUTS) project [12] began in 2011 and was the first of three projects completed in a program of laboratory medicine informatics standardisation led by the RCPA but which had active involvement from many organisations and individuals. The program is now called PITUS (Pathology Information Terminology and Units Standardisation) – a fourth phase has been planned but awaits funding. The consensus standards developed in PITUS have been endorsed and published by the RCPA as college policy – the version at the time of writing was called Standards for Pathology Informatics in Australia (SPIA) v3.0 [14], [15]. It includes requesting and reporting terminology including preferred Australian terms, standardised units, safe report rendering, information models and harmonised RIs and best practice guidance for safe laboratory medicine requesting and reporting. One of the six sub-projects for PITUS 16 was a trial implementation of an Informatics EQA (IEQA) program, therefore the purpose of this study was to evaluate a small scale pilot implementation of an IEQA program in order to study the feasibility for large scale implementation. This sub-project endeavoured to create a system to perform quality assurance on the electronic laboratory message when the laboratory sends a result back to the EQA provider itself. This was a follow-on to a previous (PITUS-14) sub project which investigated a more manual evaluation of instances of the implementation of standards for requesting and reporting working with the largest private and largest public laboratories in Australia and with active co-operation from the Medical Software Industry Association and the National EHealth Transition Authority [8], [11], [16].

A trial implementation of the IEQA program

Compliance and standardisation of laboratory medicine terminology are needed to maintain integrity of data shared between sending (laboratory medicine providers) and receiving (physicians, MyHealth Record, registries) organisations’ digital health information systems. The RCPA PITUS 16 Project Working Group 6 collaborated with RCPAQAP to design and analyse a system for reporting data using an IEQA program, the architecture of which is outlined in Figure 1.

Figure 1:

High-level IEQA architecture. IVD, in vitro diagnostic; MCT, message conformance testing.

In 2015, as part of the RCPAQAP Liquid Serum Chemistry (LSC), program, laboratories were invited to supply a routine paper report displaying results. The LSC program is a commutable frozen patient serum program used to assess method differences. The RCPAQAP then analysed these reports against the SPIA (formerly known as the RCPA APUTS v2.3 standard) [28], and variations were identified [29]. This provided validation for the rationale for development and trial of an IEQA protocol to test compliance of existing Health Level-7 (HL7) reporting standards with reporting standards developed by the RCPA [1], [30]. Accrediting bodies such as the National Association of Testing Authorities, Australia (NATA) could then use this to assist with compliance. Medical Objects Pty Ltd [31], a medical software vendor, was selected after formal evaluation of responses to a call for expressions of interest. The evaluation software was co-designed. The system built was capable of sending standardised electronic request messages and receiving electronic report messages and then analysing the message received. Message services supported Secure Message Delivery (SMD)-based secure messaging. Two laboratories volunteered to send HL7 v2 report messages with atomic clinical chemistry results for the RCPAQAP LSC program. These sites represented two of the major laboratory information systems (LIS) in use in Australia and serviced both hospital and community patients and hence were sending results to multiple Hospital Information Systems and General Practitioner practice systems. The Medical Objects’ software tool was used by RCPAQAP to test compliance of the HL7 v2 report messages received from each laboratory against the HL7 Messaging Standard Version 2.4 [32], Australian Standard AS4700.2:2012 [33] and RCPA APUTS v2.3 standards [34] (an earlier version of SPIA [28] but with most of the same compliance points). A mock RCPAQAP Informatics Program Survey Report was designed and then compiled for each of the laboratories, reporting on an assessment of the validity, integrity and rendered form of the data received.

Key milestones of the IEQA trial

Key milestones of the IEQA trial are listed and described in Table 1.

Table 1:

Key milestones of the IEQA trial.

Key milestone	Stages
1. Development of the software by Medical Objects. Two new software modules for the trial implementation, including:	1. Multi-component test requests (e.g. liver function test) (electronic requesting of clinical chemistry request orders) 2. Quality assurance-compliance rule checking module for HL7 v2.4 report messages against HL7 Messaging Standard v2.4 and AS4700:2:2012 standard; atomic data in HL7 v2.4 report messages against SPIA, including checking LOINC code, preferred term, reference interval, flagging, alignment and units
2. Installation, setup and verification of the system software and communication services	a) Medical Objects “Explorer” software application installed on computers used for compliance testing b) Medical Objects’ “Eclipse” communication services used during the trial implementation to electronically send HL7 v2.4 request messages to the participating laboratories as well as receive HL7 v2.4 report messages from the participating laboratories
3. Implementation of the IEQA program and compliance testing of the received HL7 v2.4 report messages	a) Using bulk orders module, a clinical chemistry test request for Liquid Serum Chemistry program was created and an electronic HL7 v2.4 request was electronically transmitted to the two laboratories. Also sent a PDF version of clinical chemistry request form via email b) The participating laboratories electronically transmitted HL7 v2.4 report messages with the results to RCPAQAP for analysis c) Quality assurance module performed compliance rule checks on each received HL7 v2.4 report messages. Assessed  a. Compliance of HL7 v2.4 report message against the HL7 Messaging Standard v2.4 and AS4700.2: 2012 standard, including compliance to conformance points in the AS4700:2:2012 standard  b. Compliance of atomic result data in the HL7 v2.4 report messages against terminology standards (LOINC code, preferred term, reference interval and units) and harmonised reference intervals described within SPIA [15] (Figure 2)  c. In the quality assurance module, windows were also provided for the tester to perform manual comparison of the rendered clinical chemistry report against the expected SPIA format, which is important for certain SPIA standards that require manual checking (Figure 3)
4. Compilation of a draft Informatics Program Survey Report	a) These were provided to each laboratory that participated, to assist them in identifying compliant areas and areas requiring further improvement
5. Review of trial implementation	a) Presentation of report showing the compliance checking tool to each participating laboratory

1. LOINC, Logical Observation Identifiers Names and Codes.

Issues around the implementation

In the IEQA trial, the participating laboratories were sent and could receive the electronic request message, but their LIS were not configured to process it. Electronic laboratory requesting was not the main focus of the trial and although electronic laboratory requesting is technically possible, more work is required by the RCPAQAP to find the best solution for how to communicate the 60+ systematized nomenclature of medicine-clinical terms Australia codes for the individual analytes of the RCPAQAP LSC program in an electronic request message. Participating laboratories would also need to configure their laboratory information system to process these codes.

As for other EQA programs the IEQA still needs a subject matter expert to be involved in running the EQA round. In this case that means a laboratory medicine informatician familiar with messaging standards and reporting requirements.

Figure 2:

Example of compliance rule check for the atomic result data against the RCPA published SPIA.

Figure 3:

Example of quality assurance software windows, with report format using SPIA rendered report rules (left) and rendered clinical chemistry report from the HL7 v2.4 report message (right).

The lack of a significant industry driver to encourage laboratories to configure their LIS to receive a standardised electronic request message remains a barrier. This barrier would be overcome if there was an IEQA in place to identify those laboratories that were not using the SPIA standards.

Another issue is that patients have the freedom to select which collection centre they have their samples collected, and that collection centre may not belong to the laboratory suggested by the requestor. The lack of a centralised laboratory medicine order message broker means there is no direct link between requestors (e.g. physicians) and the laboratory performing the service. The adoption of standards for laboratory medicine informatics by the clinical chemistry accreditation scheme [National Pathology Accreditation Advisory Council (NPAAC)] and/or for requesting a common request hub would drive standardisation in the right direction.

Conclusions

For laboratory medicine services to provide quality post-laboratory services to clinicians and patients, it is essential that programs are in place to ensure ongoing proficiency of test result reporting as well as standardisation of test results. Removing barriers to interoperability, both between sending (laboratories) and receiving (clinicians, MyHealth Record, patients or registries) organisations is particularly important with the implementation of laboratory medicine reports into electronic health records. This paper demonstrates the feasibility of an IEQA program supported by ICT which has the potential to be used by accrediting bodies to assist with compliance with PITUS standards and HL7 v2 messaging. This is a multi-step process which first drives standardisation and that in turn reduces variation which in turn reduces error and thereby harm. Expansion of this IEQA program to a large scale implementation across Australia will reveal its true benefits in improving communication, standardisation and patient safety in laboratory medicine in the era of electronic health records.

The described IEQA model could be used in any country where there is electronic transmission of requests and results. In Australia, there are guidelines for the format of reports and the transmission of results, these would need to be in place as well. This is a key initiative to reduce this under-recognised post-laboratory error. We believe that EQA providers in each country could develop a similar IEQA in the interests of patient safety.