Can inter-observer consistency be achieved in the laparoscopic assessment of the peritoneal carcinomatosis index score in peritoneal metastasis? A pilot study

Introduction

The peritoneal carcinomatosis index (PCI) is a scoring system introduced by Sugarbaker [1] in 1995 to quantify the extent of malignancy within the peritoneal cavity. Initially validated for assessing the spread of peritoneal metastasis (PM) in gastrointestinal malignancies [2], the PCI has since been broadly applied to PM of various etiologies, particularly during laparotomy [3], [4], [5], [6]. The PCI is calculated by summing scores from 13 abdominopelvic regions (nine abdominal and four intestinal quadrants), based on the size of tumor implants and the presence confluence with individual region scores ranging from 0 to 3, yielding a total score between 0 and 39.

While the PCI is well-validated for laparotomy, preoperative assessment is essential to avoid unnecessary cytoreductive surgery. Laparoscopy has demonstrated a negative predictive value for complete cytoreduction surgery ranging from 54 to 96 %, and for optimal cytoreduction (tumor residue <1 cm) from 69 to 100 % [7] with positive predictive values between 66 and 98 % [8]. A study by Angeles et al. [9] reported good concordance between laparoscopic and laparotomic PCI evaluation.

However, visual PCI assessment can be subjective, varying with the surgeon’s experience and interpretation. Few studies have investigated the reproducibility of PCI scoring across different surgeons. The primary aim of this study was to evaluate inter-observer reproducibility of the laparoscopic PCI score and to identify factors influencing this variability.

Methods

Results

A total of 25 laparoscopic videos were reviewed, with surgeons having a median of 6 years of surgical experience [4–22], and 5 years [2–17] specifically in PM surgery (Table 1). The median patients age was 63 years, with 64 % having undergone previous abdominal surgery, and 88 % receiving neoadjuvant chemotherapy. The predominant cancer origins were ovarian (36 %) and colon (32 %), but also included gastric, breast, rectal, and small intestine origins (Table 2).

The median PCI score was 12/39 [0–39], with 10/13 [1–13] regions typically visualized (regions 9–10 were the less visualized) a median adhesion score of 3/10 [0–10] (Table 3). The total PCI reproducibility yielded an ICC of 0.846 (95 % CI 0.738–0.927) with weak agreement for regions 4, 5, 8 (Light’s kappa 0.35–0.38), moderate agreement for regions 6, 7, 12 (Light’s kappa 0.41–0.58) and strong agreement for the other regions (Table 4).

A history of abdominal surgery significantly influenced inter-observer agreement (p=0.029), while age, BMI, primary cancer type, history of PIPAC, and neoadjuvant chemotherapy did not. Table 5 illustrates the variability in PCI score estimates across observers, compared to the median PCI calculated from all assessments. No statistical difference was observed between the observers (p: 0.07) (Table 5, and Figure 1). Also, no correlation was observed between the surgeons’ length of experience in peritoneal metastasis surgery and the variability in PCI estimates.

Figure 1:

Difference between each surgeon’s PCI estimate and the mean PCI estimate across all surgeons. Each color represents a surgeon.

Discussion

In our study, we observed a we observed a substantial inter-observer ICC of 0.846 (95 % CI 0.738–0.927). These findings are close to those reported in previous studies. For example, Gouy et al. [11], in a study involving 29 patients with ovarian PC and 16 surgeons (both junior and senior surgeons), reported a slightly higher ICC of 0.92 (95 % CI 0.86; 0.97) when evaluating PCI via laparoscopy. They found no significant difference in PCI assessments between senior and junior surgeons, nor between evaluations conducted via laparoscopy and laparotomy by the same surgeon. Similarly, Elias et al. [12] demonstrated concordance rates of 0.89 and 0.88 between experienced and less experienced surgeons in assessing total PCI and the number of affected regions, although in their study, PCI was evaluated by laparotomy.

In our clinical practice, video recordings during diagnostic or feasibility laparoscopies are left to the operator’s discretion. As a result, not all patients undergoing laparoscopy were included in this study. However, 25 high-quality videos were randomly selected between November 2020 and 2022 from two high-volume PM centers.

The inclusion of patients with varying origins and severities of PM, and a median PCI score of 12/39 (0–39), reflects a real-world patient cohort. A median of 10/13 regions were visualized, underscoring the feasibility of obtaining a comprehensive evaluation of the peritoneal cavity via laparoscopy. Furthermore, the surgeons participating in the study represented a broad range of ages and experience levels across two teams. There was no statistically significant difference between the surgeons, with the maximum mean discrepancy being 1.1/39 points. This small difference is unlikely to affect the decision to perform cytoreductive surgery in real life practice, particularly for ovarian carcinomatosis, where a PCI threshold for surgical eligibility has not been clearly established.

Cytoreductive surgery combined with intraperitoneal chemotherapy for carcinomatosis carries morbidity rates between 12 and 52 % and mortality rates of 0.9–5.8 % [13]. Therefore, accurate patient selection for this aggressive surgery is crucial. Despite advancements in preoperative imaging modalities like CT, MRI, and PET, their sensitivity for detecting lesions smaller than 5 mm remains limited [14]. The PCI score remains one of the most reliable tools for objectively quantifying disease extent [6]. Preoperative laparoscopic assessment of PCI offers several advantages, including shorter procedure time, faster postoperative recovery, reduced pain, and lower morbidity. The teams in Angers and Nantes frequently conduct feasibility laparoscopies within 15 days of planned cytoreductive surgery to streamline operating room logistics and spare patients from unnecessary surgery if resectability is not feasible.

In our study, 88 % of patients had received neoadjuvant treatment, which did not explain the variability in PCI estimates among the surgeons. According to Rawert et al. [15], the PCI score remains a reliable predictor of complete cytoreductive surgery even after neoadjuvant chemotherapy, with an AUC of 0.77 [(95 % CI 0.53–1.00), p=0.050].

However, laparoscopic PCI assessment can face challenges, particularly in the presence of adhesions that limit visualization of certain regions. In our study, prior abdominal surgery contributed to variability in PCI assessment, with a median adhesion score of 3/10. Von Breitenbuch et al. [16] reported than in 71 % of 102 diagnostic laparoscopies, the PCI could not be fully evaluated due to inadequate exploration of the peritoneal cavity, and four procedures had to be converted to laparotomy due to significant adhesions obstructing laparoscopic exploration.

This raises the question of whether certain regions are more critical to assess than others in cases where the peritoneal cavity cannot be fully explored [17]. Heitz et al. [18] noted that primary recurrence sites for ovarian carcinomatosis include the small intestine, the mesentery root, and the hepatoduodenal ligament (regions 2 and 9–12), suggesting that incomplete preoperative assessment could lead to suboptimal initial cytoreductive surgery. Rosendahl et al. [19] similarly found that failure to achieve CC0 resection in 80 % of cases was due to involvement of regions 2 and 9–12. They further demonstrated that these regions were more predictive of complete surgery than the total PCI score (AUC of 79 vs. 75 % for total PCI) and were associated with poorer survival outcomes (HR: 1.08; 95 % CI: 1.06–1.11). Access to these regions can be challenging during laparoscopy, potentially leading to disease underestimation.

Improving reproducibility in PCI assessment would enable a more objective evaluation of PM, fostering a standardized approach that minimizes surgeon subjectivity. As Artificial Intelligence becomes increasingly integrated into surgical practice, establishing standardized criteria for laparoscopic PCI evaluation – such as the number or location of regions to be explored – could significantly enhance accuracy and consistency.

Conclusions

This study highlights the substantial reproducibility of PCI assessment via laparoscopy, emphasizing the impact of prior abdominal surgery on visualization and scoring. The ICC of 0.846 suggests a high level of agreement among observers, reinforcing the reliability of laparoscopic PCI evaluation. Further research is required to refine laparoscopic techniques, improve inter-observer consistency, and correlate visual macroscopic evaluations with histopathological findings. While no universally accepted PCI threshold has been established, improving the accuracy of PCI assessment may help identify specific thresholds that could provide valuable guidance for selecting candidates for cytoreductive surgery and intraperitoneal chemotherapy.