How to validate radiopharmaceuticals management software?

Background

In the context of securing medication management system (MMS) in healthcare facilities, implementation of an electronic MMS is a priority area of work. This major development decreases the risk of error but could induce new types of errors [1], [2].

Radiopharmacy is a discipline of hospital pharmacy dealing with radiopharmaceuticals that will be administrated to patients to diagnose or treat specific diseases in nuclear medicine.

The French radiopharmaceutical medication use process has eight subsystems: management/supply; prescription of the radiopharmaceutical; pharmaceutical analysis of the prescription; pharmaceutical validation of the prescription; preparation/quality control/release; preparation of the individual patient doses and administration [3], [4].

The quality requirements for the production of radiopharmaceuticals have increased significantly. The deployment of Information Technology (IT) tools is essential to secure the radiopharmaceutical management system [5], [6].

In France, prescribing and dispensing software were subject to a certification obligation. On December 7, 2017, the European court of justice has decided to no longer make compulsory the certification obligation for drug prescription assistance software and dispensation assistance software. An European conformity (CE) marking on the prescription support software is now imposed [7].

Currently, there is no radiopharmacy software bearing the CE mark in France.

Furthermore, software validation is increasingly mentioned by French authorities’ in inspection reports. A working group has been created within the FRench Society of Radiopharmacy (SoFRa) to define specific requirements for securing MMS. A standard specification for radiopharmacy software has been defined [8]. This document describes radiopharmacy software in their environment.

Radiopharmacy software could be integrated in a global solution with appointment scheduling software and radiology information system (RIS) or could be interfaced to them. They also have to be interfaced to different materials or applications: dose calibrator, radio thin layer chromatography (TLC) scanners, dose archiving and communication system (DACS)…

This work focused on: software supporting the radiopharmaceutical management system; equipment software (dose calibrator, automated dispensing software, radio TLC scanners) and software interfaces (appointment software, DACS, etc. …).

Little data exists in the literature on software validation in hospital pharmacy [9], [10]. The validation of software in radiopharmacies is not formalized because of the lack of tools.

A consensual tool to validate the medication use process is essential for radiopharmacists to improve patient safety.

The aim of this work is to propose the first consensual tool to validate radiopharmacy software approved by a panel of experts.

Methods

The Delphi method was chosen to highlight a convergence of opinion in an area of expertise in order to reach a consensus [11], [12]. A questionnaire is sent to a panel of experts and responses are anonymous. Each answer was scored by the experts to indicate their level of agreement with the criteria. Data and comments were analysed and new proposition were done according to the experts. The process was repeated several times until a consensus was reached.

The requirements were proposed by a three-members working group (two radiopharmacists and a resident) and submitted for validation by a 2-turn Delphi method.

Experts were selected to represent radiopharmacists and pharmacists involved in software validation. To represent various healthcare facilities, experts from private and public sector were selected. Surveys (Google forms) were sent by mail to radiopharmacists, pharmacists responsible for validating electronic MMS in hospital pharmacy and radiopharmacy residents.

A text was accompanying the survey to explain the problem of software validation in radiopharmacy, the aim of the study, the planned deadline for responses and the Delphi method. The scope was also defined in the text: software system managing kits, generators, radiopharmaceuticals, radiopharmaceutical preparations and radioactive implantable medical devices and their interfaces.

The first survey included following questions: occupation, number of years in practice and conflicts of interest of the experts.

The working group put forward 56 issues gathered from multiple documents analysis. A literature review was conducted on the validation of hospital pharmacy software. The good manufacturing practices were also studied [13]. The initial proposal was also based on various requests from the authorities: Regional Health Agency (ARS), Nuclear Safety Authority (ASN) inspections report and French Health Authority (HAS) certification.

The first questions were related to personals who must be implicated in the validation.

Questions concerning alerts for securing MMS, mentioned by the SoFRa [8] were proposed for approval.

The working group also adds several parameters which must be essential during all the eight steps of the MMS: management/supply; prescription of the radiopharmaceutical; pharmaceutical analysis of the prescription; pharmaceutical validation of the prescription; preparation/quality control/release; preparation of the individual patient doses and administration.

Open-ended questions were asked to allowed experts to freely express themselves: comments on the given criteria, criteria to be added, plan and design of the tool, others remarks.

Each criterion was scored on a nine-point Likert scale ranging from 1 (not at all relevant) to 9 (very relevant) according to the RAND Foundation (Research ANd Development).

Consensus was considered to be reached if more than 70% of the experts agreed on an item (score between 7 and 9) and less than 30% disagreed (score between 1 and 3).

The experts had 3 weeks to complete each round. Reminders were sent 1 week before the end of the survey. Figure 1 summarizes the chronology of the study.

Figure 1:

The chronology of the study.

Results

Results of the survey

First round

Thirty-six experts were asked to complete the first survey (24 radiopharmacists, eight hospital pharmacists, four radiopharmacy residents). Fifty-six questions were associated with criteria defined by the working group. They were sent with a 3-weeks deadline to reply: 27 replies were recorded (77%). All categories of experts were represented. Seventy-eight percent of the experts were radiopharmacists. Thirty-three percent had between 11 and 15 years of experience. The average response time was 15 min.

The median scores for each item except one were greater or equal to 7 with more than 70% between 7 and 9 and less than 30% of the scores between 1 and 3. The consensus was therefore reached from the first round except for the question “who must be involved in the validation of radiopharmacy software?”

Some of the experts dissented on the involvement of medical physicist, radiation safety officer, pharmacy technicians, nuclear medicine physician and computer scientist but not on X-ray technicians, radiopharmacy resident, pharmacist (not radiopharmacist), pharmaceutical company and software and interfaces company.

A total of 16 comments were gathered in the open-ended questions. Most of the comments of the first round concerned frequency of validation, so we added those criteria in the second survey for each question. The items of the two surveys were the same.

Second round

The second survey was based on the answers of the first survey. The items were classified into four groups by the working group: general part, alerts, data retention/printing and interfaces. The questions with different parts were also simplified. For example, the five following questions “Is a report modification traced (name, quality, time and date) for the following items: batch file, order, prescription book, pharmaceutical release, pharmaceutical validation?” are merged into one question in the second survey.

The second survey was composed of 44 questions and deals with the frequency of validation. Experts have to decide when the test must be performed:

1. after the software installation (initial check)

2. after update by the supplier

3. after a change of configuration (new radionuclides, radiopharmaceuticals, protocol …)

4. and/or periodic.

Seventeen responses and eight comments were recorded.

The comments about responsibility of the professionals and communication problem between professional involved in validation were reported in the open-ended questions. Degraded modes in case of computer failure were also mentioned by the experts (available paper register …). Sufficient time have to be allocated to software validation. Training on software was an important point for experts. They mentioned the supervision, the traceability, the validation of those trainings and e-learning and paper documentation as a support. A global validation is a heavy workload and the experts recommend choosing the tests that have to be performed for each situation.

There was a consensus on each criterion except on frequencies of the verification of each item for the periodic validation. Each organization is different and a consensus is difficult to reach. So, a third round was not sent just for this item.

The 44 items validated by the experts are in Figure 2.

Figure 2:

Validation of software system managing kits, generators, radiopharmaceuticals, radiopharmaceutical preparations and radioactive implantable medical devices and their interfaces: consensual items.

Discussions

Software validation in hospital pharmacy is generally not formalized. Each radiopharmacy is organized in a different way according to nuclear medicine exams, treatments realized and patient population (paediatrics). Radiopharmacy could also use different software (four French software publishers in the fields of radiopharmaceuticals) and staff could be pharmacy technicians or X-ray technicians. This study only deals with radiopharmacy management software and their interfaces. If the radiopharmacy management software is integrated in nuclear medicine management system, this grid will be a part of the validation of the system. The interfaces part had also to be modified to delete superfluous questions.

The validation tool should be implemented in all French radiopharmacy so we used a consensual method. The particular advantage of the Delphi method is to avoid pitfalls of direct confrontation and influence among experts. They could express themselves anonymously. Furthermore, this mode of working enables consultation of experts very widespread geographically. This cheap method is also easy to implement. The drawback is the lack of opportunity for dialogue among participants. To obtain a quick consensus on a validation grid, this method was a good option.

The experts were selected for their knowledge in the field of radiopharmacy (radiopharmacist, resident) or the electronic MMS (pharmacists responsible for validating electronic MMS in hospital pharmacy). The limit of the tool is that some criteria fall outside of the field of competence of pharmacist or radiopharmacist (IT department, nuclear physician and medical physicist).

Although they may not be applicable in French radiopharmacies, the good manufacturing practice were analysed to propose relevant issues in the first survey. Alarms identified by the SoFRa had also to be validated by the experts. The SoFra analysed the risk maps of different hospital structure to defined alerts that must be set up in radiopharmacy software. It was important to validate this part by the experts.

The high rate of response shows the interest of the experts in this study. In the first round, experts had approved unanimously practically each item. The questions chosen by the working group seem to be relevant. The second round was to propose a more accessible grid by synthetizing and classifying items.

The main comments of the experts concerned the implementation of the validation. A trial version of the software must be installed before putting in into production. This allows various tests without affecting the IT environment.

Different types of controls have to be completed by many stakeholders. We recommend that each radiopharmacist define, according to its internal organization, the responsibilities of the different actors involved. A protocol has to be drawn up before validation to define responsibility and criteria that could impact daily work. A risk rating must be established to assess risk situations [14].

Each criteria can be classify as follows:

1. Level 1: low risk, no impact on daily work

2. Level 2: low risk, impact on daily work

3. Level 3: high risk, to be resolved immediately and use of software must be stopped

4. Level 4: unacceptable risk and use of software must be stopped

MMS are not “set and forget” systems. Once implemented, MMS require substantial ongoing management, updating, refinement and improvement [15], [16]. Then, the software publisher must be informed of the test results. A new validation must be perform to test each new software version [17].

The periodic validation remains subject to debate. His frequency depends on the time allocated to software validation.

A few proposals were relating to the safety of the radiopharmaceutical medication use process in case of electronic systems failure. Each healthcare facility has to make their own procedure in case of computer failure (prescription pad, blank documents and registers/records).

This complete tool will help radiopharmacists to improve quality by putting in place all the requirements of the grid.

Conclusion

Pending the establishment of software CE marking and/or certification, improvement of radiopharmaceuticals management system must be performed.

We proposed the first consensual tool to validate radiopharmacy software. Each institution should define, according to its internal organization, who (IT, pharmacy, etc.) and when (frequency) the validation must be carried out. These consensual criteria will help to secure radiopharmaceutical management system and improve patient safety.

We would like to thank all survey participants for their time.