Assessing data completeness in the international society for the study of pleura and peritoneum (ISSPP) PIPAC database: a multicenter evaluation from 2020–2024

Introduction

Peritoneal metastasis (PM) is a common condition in patients with gastrointestinal and gynecological cancer. Systemic chemotherapy tends to have a relatively short effect in these patients, reflected in their poor prognosis [1], 2]. A decade ago, Pressurized Intraperitoneal Aerosol Chemotherapy (PIPAC) was introduced to overcome the limitations of systemic chemotherapy during palliative treatment in patients with PM [3]. PIPAC reached stage 2b of the IDEAL framework in 2020, and a prospective international PIPAC database was launched by the International Society for the Study of the Pleura and Peritoneum (ISSPP) and is hosted by Odense Patient data Explorative Network (OPEN) [4], 5]. The ISSPP PIPAC database (PIPAC database) has been implemented using REDCap (Research Electronic Data Capture) [6], a web-based software solution for easy online access, and consists of six key elements: Patient, Consent, Treatment, Complications, Response evaluation, and Follow-up. The results from the PIPAC database have been described in two previously published annual PIPAC reports [4], 7], and the PIPAC database is considered essential for monitoring indications, complications, and potential clinical effect, while awaiting data from randomized controlled trials.

However, most international databases face challenges during initial construction (e.g., mode of access, implementation of an easy-to-use interface, costs) and continued operation (e.g., maintenance and monitoring costs), as well as data quality and validity (e.g., inaccurate, incomplete, or missing data). The latter may have a negative influence on the final data analyses and conclusions [8], 9]. In 2023, the second annual PIPAC database report showed missing and incomplete data on specific topics like date of death, reasons for stopping PIPAC, and nonperitoneal regression grading score (non-PRGS) response evaluation [7].

As continuous monitoring of missing and incomplete data is an important part of qualifying and improving the database reports, the ISSPP Registry Group decided to evaluate this before publication of the third annual PIPAC database report. Thus, the primary study aim was to assess data completeness within the ISSPP PIPAC database. Secondary objectives included identifying patterns of missing data and proposing improved strategies for data entry, monitoring, and reporting.

Methods

A retrospective analysis of data completeness within the ISSPP PIPAC database was performed on data exported on 14th of October 2024, which included all data entered since the official launch in June 2020. The primary outcome was to investigate the number of reporting centers with incomplete data in the six key elements. Incomplete data, for this specific purpose, was defined as missing or incorrectly registered data in more than 50 % of the recorded patients within each reporting center. Incorrect data registration was defined as data entered, but not in accordance with the guidelines provided by the ISSPP Registry Group, which are available on the official ISSPP website [10]. Based on this initial data completeness analysis, we evaluated reasons for incomplete data, including potential errors or mistakes owing to the structure and/or presentation of the database (labels, annotations, etc.), and, where relevant or possible, provided potential remedies.

Further, an in-depth analysis of Response Evaluation, Complications, and Follow-up were performed to assess the extent of missing data in these three key elements. These elements were chosen due to their significance with regard to reporting PIPAC data, and because these elements had missing data in the second annual PIPAC report [7]. Missing data were defined as no registered data due to either lack of data or incorrectly registered data (Figure 1). The in-depth analysis of complications included an examination of the centers that reported most complications and the frequency of severe complications per PIPAC center.

Figure 1:

Primary outcome of the study and in-depth analysis of Response evaluation, Complications, and Follow-up. *Incomplete data were defined as either missing or incorrectly registered data in more than 50 % of the recorded patients. **Missing data were defined as no or incorrectly registered data.

The governance, legal aspects, ethical framework, variables, and implementation of the ISSPP PIPAC Database have been previously described in detail [4], 7] (Table 1).

Results

Primary outcome

As of October 14th, 2024, the ISSPP PIPAC database contained data from 2,777 PIPAC procedures in 950 patients. Thirty PIPAC centers were registered in the database, and 16 (53 %) centers had actually included patients. Incomplete data were observed in four of the key elements, but not in Treatment and Patient. Most incomplete data were in Complications, Response Evaluation, and Follow-up (Table 2). Six of the 16 reporting centers (38 %) had no incomplete data, and no reporting center had incomplete data in all key elements.

Table 2:

Reporting centers (n=16) with incomplete data^a in the six key elements.

Key element	Number of reporting centers with incomplete data, n (%)
Patient	0 (0)
Consent	1 (7)
Treatment	0 (0)
Response evaluation	5 (31)
Complications	7 (44)
Follow-up	2 (13)

1. ^aIncomplete data were defined as missing or incorrectly registered data in more than 50 % of the recorded patients within each reporting center.

All centers had complete data in Patient, Consent, and Treatment elements – except one center where the majority of patients did not consent to having their information passed on to the database.

Reasons for incomplete data in Response Evaluation were due to missing data or incorrect registration of data, see below (5 centers).

Complications had the highest amount of incomplete data (7 centers). This was due to incorrect registration of data. The major cause being centers incorrectly marking new complications without registering further data and thereby creating “empty” complication records^[1] (Figure 2). For a more detailed description of this topic, please read below.

Figure 2:

Incompleteness in the key element Complications. The patient was treated with eight PIPAC directed treatments. As highlighted in red, eight complications were registered but only one complication was provided with information as to the character of the problem (e.g., bowel injury). Seven of the complications had no registered data and are considered “empty.”

In Follow-up, incomplete data (no follow-up) were observed in two (13 %) reporting centers, only.

Discussion

The voluntary, noncommercial ISSPP PIPAC database has included PIPAC centers and patient data for more than 4 years and two annual reports have been published [4], 7]. Currently, 16 centers are reporting their data on PIPAC procedures to the ISSPP PIPAC database as of October 2024. Continuous monitoring of data is relevant for several reasons (e.g., database design, use, variable definitions, and reporting), and assessment of data completeness and accuracy are important criteria for this evaluation. This study assesses data completeness within the ISSPP PIPAC database and represents the first systematic audit of a prospective international PIPAC registry.

Incomplete data, when defined as either missing data in more than 50 % of the included patients or incorrectly registration of data in more than 50 %, were found in four of six key elements of the database. The key elements were Complications (7/16 reporting centers), Response Evaluation (5/16), Follow-up (2/16), and Consent (1/16). Data were complete in the key elements Patient and Treatment.

The incomplete data in Consent resulted from one center, where most patients did not consent to having their information registered in the database. Consequently, data from these patients were excluded from this study’s analysis.

Reasons for incomplete data in Complications and Response evaluation were mainly incorrect registration of data, and the reporting centers often created the same errors. In Complications, the main error was registration of complications without providing subsequent data and thereby creating “empty” complication lists. The misunderstanding seems to arise when investigators believe empty complications mean no complication has occurred. Thus, important to emphasize in the available database guidelines, that when pressing “yes” to the variable “complication (yes/no)” under the key element Treatment, the investigator must register the new complication separately under the key element Complication. If “no” is chosen, then no complications occurred, and no separate complications should be registered.

One error in registering complications have previously been a misuse of the “other”-field, with some centers listing numerous complications in the text field. However, this was not observed in this data overview.

Regarding Response evaluation, the correct method to register data is to select the type of histological response evaluation that was performed during the specific PIPAC directed treatment under the key element Treatment (Figure 3). As underlined in the guideline, the investigator should not create a separate response evaluation, if they choose “none.” On the other hand, if the investigator chooses “PRGS evaluation” or “non PRGS evaluation,” it will be necessary to create a response evaluation (Figure 4).

Figure 3:

Under the key element treatment, the investigator should choose which kind of histological response evaluations is performed. There are three options: None, PRGS evaluation, or non-PRGS evaluation.

Figure 4:

Creation of a new field under the key element Response evaluation. The investigator chooses whether PRGS or non-PRGS was performed and is prompted to provide additional response data.

In the Patient element, one center used initials instead of recognizable numbers as patient-ID. Thereby the center risks that two patients are included with the same initials, and therefore it cannot be recommended to use initials as patient-ID. This will be emphasized in an updated database guideline.

The in-depth analysis of Complications, Response Evaluation, and Follow-up revealed missing data in 72 % of patients with non-PRGS response evaluation. Because of overlapping between the two categories “no data” and “incorrect data,” it is difficult to differentiate whether the high number of missing data is caused only by incorrectly registration or that some investigators forget to register the non-PRGS response. The same argument applies for the missing data in Complication and Follow-up elements. However, no other variables in the in-depth analysis exceeded 16 % of missing data, which must be considered very satisfactory at the present state. In comparison, 15 out of 315 patients (4.8 %) were excluded due to “incomplete data” in a recent international registry of patients with peritoneal metastases (PM) undergoing cytoreductive surgery and hyperthermic intraperitoneal chemotherapy (HIPEC) [11]. The term “incomplete data” was not further defined. Direct comparison between studies is challenging, since the present study evaluates incomplete data on a variable level, whereas the HIPEC study focused on assessment on patient level. A conservative estimate of patients excluded from the ISSPP PIPAC database due to “incomplete data” would be less than 10 %. On a variable level, one of the major databases including 492,242 patients with invasive breast cancer, the clinical stage information was missing in 29 % of the patients [12]. In that perspective, the current extent of incomplete data in the ISSPP PIPAC database seems acceptable.

It is well known that major multicenter databases must accommodate incomplete data, and that achieving a complete dataset for all included patients is unlikely [8], 13]. Missing data may have quite different clinical implications depending on the topic of the database. One study of missing data in a national health database in the United States concluded that a high prevalence of missing data was associated with differences in overall survival. This emphasizes the importance of complete documentation and registration [8]. Data completeness also depends on the dedication and resources of the involved centers, the hosting center, analysis group, and affiliated clinical societies.

Lack of data completeness is also a well-known problem in both pro- and retrospective PIPAC studies. A systematic review on quality of life in patients treated with PIPAC found moderate risk of bias due to missing data in six of nine included studies [14]. A narrative review on response evaluation in PIPAC detected a substantial amount of missing data or discontinued treatments. The review showed that missing data or discontinued treatments in more than half of patients occurred in 62 % of the included studies [15]. Similarly, this critical database evaluation detected 31 % of the reporting PIPAC centers having incomplete data in the key element Response evaluation. Absence of response evaluations may affect interpretations of PIPAC safety and efficacy, potentially biasing clinical decision-making or outcome reporting.

A retrospective study on safety of PIPAC combined with other surgical procedures reported that only 6 % of the patients have been excluded due to missing data, but this is probably due to selection [16]. Another systematic review on reasons for stopping PIPAC showed that missing data ranged from 0 to 24.4 % among the included studies [17]. These numbers are in line with the present study, where reasons for stopping PIPAC were missing in 15 % of the patients. Furthermore, the systematic review reported that one of the main limitations of the study was missing data [17]. Therefore, continued investments in data collection and correct registration remain a critical step toward more complete and trustworthy real-world data – also in the ISSPP PIPAC Database.

Conclusions

International databases must deal with missing and inaccurate data, and this was confirmed in four of the six key elements within the ISSPP PIPAC Database. However, overall data completeness was considered satisfactory at the present state, and the ISSPP Registry Group has launched several initiatives to improve data completeness and quality to enrich future data sets and annual reports.