DSC: depth data quality optimization framework for RGBD camouflaged object detection

2.1 Method overview

Figure 3 presents the overall architecture of the depth selection and calibration (DSC) framework. This framework comprises two key components: evaluation-based selection and depth calibration. Specifically, we first generate depth data using various MDE methods to establish an initial database. Subsequently, based on our proposed depth quality evaluation metric (Quality score, represented by Q), we select the optimal depth map for each image from the initial database to construct a preliminary RGBD COD dataset. Finally, we employ the designed Two-stage Depth Calibration (TDC) strategy to calibrate the depth maps in the preliminary RGBD COD dataset, correcting potential noise introduced by unreliable original depth maps, thereby constructing a high-quality RGBD COD dataset. In the following sections, we will discuss these two components in detail.

Figure 3:

Detailed architecture of our depth selection and calibration (DSC) network (D-A-v2 represents Depth-Anything-V2).

2.2 Evaluation-based selection

As previously discussed, depth data generated by different MDE methods exhibits significant variations, and researchers typically rely on subjective observations to assess depth quality, which is both time-consuming and difficult to quantify. Moreover, due to the varying complexity of camouflaged image scenes, single MDE methods have limited generalization capability and struggle to generate high-quality depth maps consistently across all images. Therefore, relying solely on depth data generated by a single MDE method as a training benchmark cannot guarantee optimal model performance.

To address the challenges of quantitative depth quality assessment and data variability, we construct depth quality evaluation metrics from multiple dimensions to achieve quantitative assessment of depth map quality and selection of high-quality depth maps. Our proposed quality assessment metric (quality score, represented by Q) comprises five key components: structural similarity, edge consistency, foreground smoothness, depth value utilization rate, and depth difference between foreground and background.

Structural Similarity (SSIM) [23] is a widely used metric for measuring similarity between images. We compare the depth map with the grayscale version of the RGB image, using SSIM to evaluate their consistency. SSIM models similarity as a combination of three factors: luminance similarity, contrast similarity, and structural similarity. The definition of SSIM is as follows:

where

where c 1 , c 2 , c 3 is a constant that prevents instability when the denominator approaches zero. x , y represents the input image, while μ x , μ y , σ x , σ y and σ _xy denote the corresponding means, variances, and covariances, respectively.

The edge structure of high-quality depth maps should maintain consistency with the original RGB images. However, RGB images contain rich color details that are not present in depth maps. As shown in Figure 4, although Canny edge detection results from RGB images and corresponding depth maps are difficult to compare directly, we observe that high-quality depth map edges demonstrate good consistency with the ground truth edges of camouflaged objects. Based on this key finding, we propose using the mean absolute error between depth map edges and the ground truth edges of camouflaged objects to evaluate edge consistency. The specific steps are as follows: first, normalize the depth map, then use the Canny operator to extract the edge map E _D , and finally calculate the edge consistency score E between the edge map E _D and the edge ground truth E _G . The specific formula is:

where mean ⋅ represents the mean operation.

Figure 4:

Comparative example of edge extraction results.

In the real world, object surfaces typically exhibit smooth characteristics, which should be reflected as continuous depth variations in depth maps. However, MDE methods may introduce interference, such as noise and depth discontinuities, when generating depth maps. To evaluate the smoothness of depth in foreground regions, we multiply the depth map with the ground truth mask to obtain the foreground depth map and describe the spatial distribution smoothness by calculating the depth variance in the object region. The foreground smoothness S _f is defined as:

where σ f 2 represents the depth variance in the foreground region.

As a measure of object distances in a scene, depth map quality depends not only on local matching with the original image but more importantly on the richness and effective utilization of depth information. When depth values are overly concentrated (e.g., most pixels having similar or identical depth values), the depth map will struggle to accurately characterize the three-dimensional structural features of the scene. Therefore, we introduce the depth value utilization rate as an evaluation metric to quantify the distribution characteristics of depth values in depth maps, ensuring that depth maps can fully utilize the available depth range and effectively express the scene’s depth hierarchy information. This metric is based on information entropy theory and is used to evaluate the uniformity and diversity of depth value distribution. Specifically, we first divide the depth values in the depth map into N intervals (typically N = 256, corresponding to the grayscale levels of an 8 bit depth map) to obtain a depth value histogram {n _i }(i = 1,2,…,N). Then, we normalize the histogram to calculate the probability p _i for each depth value interval:

Subsequently, based on the definition of information entropy, we calculate the entropy of depth value distribution:

where ɛ is a small exponent to prevent log0 cases.

Finally, we calculate the maximum entropy value H _max and compute the normalized depth value utilization rate U:

The value of U ranges from [0,1], with values closer to 1 indicating more uniform depth value distribution and higher utilization rate.

Furthermore, although camouflaged objects exhibit high similarity with their backgrounds in RGB images, they still maintain significant spatial differences. High-quality depth maps should effectively capture and reflect these spatial differential characteristics. Based on this observation, we propose a depth difference metric between foreground and background. To ensure assessment accuracy, rather than directly calculating the average depth difference between foreground and background, we compute the average depth difference within a 5 × 5 neighborhood around the target boundary, thereby quantifying the depth map’s ability to highlight camouflaged targets. The mathematical definition of the foreground-background depth difference D is as follows:

where μ f ⋅ and μ g ⋅ represent the mean depth values of foreground and background regions within a 5 × 5 neighborhood around point (x,y) on the target boundary, respectively.

Integrating the above five dimensions, our depth quality evaluation metric (Q) is calculated as:

where w 1 , w 2 , w 3 , w 4 , w 5 represents weight parameters used to balance the contribution ratios of different metrics, which in this paper is set to {0.7, 1, 0.2, 0.3, 2}. The determination process of the weight parameters is as follows: first, all weight parameters are initially set to 1, and the mean value of each metric across all depth images is calculated; then, each metric’s mean is divided into 0.2 to obtain the corresponding weight parameter. This method ensures that each metric has equal influence in the evaluation of depth quality. By comparing quality scores of different depth maps, we can select the optimal depth map for each image from results generated by different MDE methods, thereby constructing a high-quality initial RGBD COD dataset.

2.3 Depth calibration

Spatial information provided by depth maps and the texture-free separation of foreground and background play crucial roles in breaking camouflage. However, due to domain gaps, depth maps generated by monocular depth estimation methods contain substantial noise, resulting in unreliable depth data. Direct use of such depth data may significantly degrade the performance of RGBD COD models.

To address the performance bottleneck caused by noise, similar to [24], 25], we attempt to calibrate the original depth maps to obtain high-quality depth maps consistent with foreground objects. Two key issues need to be addressed: 1) how to distinguish between good-quality (positive samples) and poor-quality (negative samples) depth maps in the initial RGBD COD dataset. (2) how to generate calibrated depth maps that preserve high-quality portions while correcting low-quality regions. Therefore, we design a Two-stage Depth Calibration (TDC) strategy, which forms the core component of DSC. The two consecutive stages involve distinguishing positive and negative samples and generating calibrated depth maps. Figure 3 illustrates the proposed TDC strategy, with specific details as follows:

Stage 1: Positive and negative samples are separated based on the IoU [26] between prediction results and their ground truth (GT). This is based on the consideration that IoU can measure the consistency between prediction results and their corresponding GT, thereby reflecting the reliability of information contained in depth images to some extent.

Specifically, first, under-ground truth supervision, we train two encoder-decoder networks with identical architectures for RGB data and depth data separately. Here, the Resnet50 network [27] serves as the encoder, while the decoder part of U-Net [28] serves as the decoder. Then, RGB data and depth data are input separately into their respective pre-trained networks to generate camouflaged object prediction results, and IoU values between each prediction result and its corresponding ground truth are calculated, denoted as IoU_depth and IoU_RGB respectively. Finally, depth images that provide reliable cues, namely samples ranking in the top 20 % of IoU_depth and samples where IoU_depth > IoU_RGB, are selected from the initial RGBD COD dataset as the positive sample set, with the remaining images forming the negative sample set. Compared to the negative sample set, depth images in the positive sample set are more beneficial for COD, with more acceptable depth quality. The middle position of the lower half of Figure 3 shows typical examples from both positive and negative sample sets.

Stage 2: Utilize a generation network to generate depth images and calibrate original depth images using the generated images.

Specifically, we first retrain the image generation network from [29] with RGB images as input and depth maps from the positive sample set as supervision information to reduce noise in the original depth data. Subsequently, RGB images from the initial RGBD COD dataset are input into the trained generation network to obtain high-quality pseudo depth images. These pseudo depth images do not directly replace the initial depth maps (depth maps in the initial RGBD COD dataset) but are used for calibration. During calibration, we adopt a spatial weighted sum of initial depth maps and pseudo depth maps to replace the original depth maps, with weights determined by depth’s contribution to detection. As shown on the right side of Figure 3, initial depth maps and pseudo depth maps are separately input into the encoder (Resnet50) for feature extraction, followed by feature fusion through the decoder to generate spatial weights. These weights are applied to both types of depth maps to obtain calibrated depth maps, which are then input into the same encoder-decoder network used in Stage 1 for camouflaged object detection. Spatial weights are dynamically adjusted during network training, and upon completion of training, optimal weights and calibrated depth maps Depth _cal are obtained, calculated as follows:

where Depth _raw and Depth _pse represent the initial depth map and pseudo depth map respectively, and ω represents spatial weights. For better understanding, we visualize intermediate results of the depth calibration process in Figure 5. Through comparative analysis of different depth maps in Figure 5, we can draw the following observations: First, comparing the third and fourth columns, Depth _pse provides richer three-dimensional spatial layout information compared to Depth _raw . Second, comparing the third and fifth columns clearly shows that Depth _cal presents more complete scene structure than Depth _raw , with clearer target structural details. Finally, comprehensive comparison of results in the third, fourth, and fifth columns demonstrates that Depth _cal exhibits significant advantages in overall visual quality.

Figure 5:

The internal inspections of depth calibration: examples of initial depth map Depth _raw , pseudo depth map Depth _pse and the calibrated depth map Depth _cal .

3.1 Experimental setup

Datasets: To evaluate the effectiveness of the proposed DSC framework, we conducted experiments on three widely used and challenging COD datasets: CAMO [30], COD10K [31], and NC4K [32]. CAMO contains 1,250 images, with 1,000 for training and 250 for testing. COD10K is currently the largest camouflaged object dataset with high-quality pixel-level annotations, comprising 5,066 camouflaged images, of which 3,040 are used for training and 2,026 for testing. NC4K is the largest camouflaged object test set to date, consisting of 4,121 images downloaded from the internet, providing a rigorous test of model generalization capability. Following mainstream training configurations [1], [5], [6], [7], 31], we use 3,040 samples from COD10K and 1,000 samples from CAMO as the training set, while the remaining samples and the NC4K dataset are used for testing.

Evaluation Metrics: To quantitatively assess the impact of different depth data on model performance, we employ four widely-used evaluation metrics: S-measure (S _α ), F-measure (F _β ), E-measure (E _ϕ ), and Mean Absolute Error (MAE, M). Among these metrics, higher values of S _α , F _β , and E _ϕ indicate better performance, while the opposite holds true for M. For detailed definitions of these evaluation metrics, please refer to [31].

Implementation details: The framework is implemented in PyTorch and trained using an NVIDIA GeForce RTX 3090 GPU with 24 GB memory. The backbone network employs Resnet50 with parameters pre-trained on ImageNet. Input images are uniformly resized to 352 × 352 pixels, and various data augmentation techniques are applied, including random flipping, rotation, and cropping. During training, the initial learning rate is set to 1e-4 using the Adam optimizer with a batch size of 16. During inference, the encoder-decoder architecture predicts camouflaged objects in an end-to-end manner without requiring any post-processing operations.

3.2 Comparative experiments

We compare Depthcal with depth data generated by four MDE methods: DPT [18], MiDaS [19], Depth-Anything-V2 [20], and Depth Pro [21]. DPT leverages Vision Transformers for dense prediction tasks and is known for its high accuracy and strong generalization across diverse visual scenes. MiDaS emphasizes robustness by mixing multiple datasets for training, enabling zero-shot cross-dataset transfer; however, its depth maps may be less sharp in fine-structured regions. Depth-Anything-V2 is designed for scalability with large-scale data and achieves impressive performance on both indoor and outdoor scenarios, but it may require substantial computational resources. Depth Pro focuses on delivering sharp and metrically accurate depth maps with fast inference speed, though its performance can fluctuate in highly complex or ambiguous scenes. By utilizing these diverse models, we are able to comprehensively evaluate the effectiveness of the proposed framework. In addition, the performance of four advanced RGBD COD methods (DaCOD [13], PopNet [14], DAINet [9], and DAFNet [15]) are evaluated on different depth data. All methods are implemented using author-provided open-source code, with MDE methods utilizing original weights and detection methods retrained using default parameters. To ensure fairness, we employ unified evaluation protocols and code for objective assessment of prediction results.

Quantitative evaluation: Table 1 presents the quantitative comparison results. As shown in Table 1, Depthcal demonstrates the most outstanding performance in overall quality assessment, achieving a quality score Q of 0.95, significantly outperforming depth data generated by other MDE methods. Compared to other depth data sources, all detection methods achieve optimal performance when using Depthcal. Specifically, taking DAFNet as an example, compared to Depth-Anything-V2 (the second-best performing depth data), using Depthcal results in average improvements of 2.5 %, 1.2 %, and 2.2 % in Sα, Fβ, and Eφ metrics respectively, while reducing M by 0.3 %. To clearly illustrate the experimental results, we visualize part of the quantitative data from Table 1 as line graphs (as shown in Figure 6). Through systematic analysis, we find a significant positive correlation between depth data quality score Q and model performance: higher quality scores correspond to better model performance. This finding strongly validates the rationality of our proposed quality assessment metrics. Additionally, the performance variations of different detection models across various depth data sources further confirm the effectiveness of the Depthcal.

Figure 6:

Comparison of S-measure metrics across different methods on three benchmark datasets (D-A-v2 represents Depth-Anything-V2).

Qualitative evaluation: Figure 7 shows typical samples generated by Depth _cal and various advanced MDE methods along with their corresponding quality scores. Comparative analysis indicates that calibrated depth (Depth _cal ) provides richer 3D scene information and target structural details. Meanwhile, depth maps with higher quality scores typically exhibit superior visual quality and better foreground consistency with corresponding RGB images. Figure 8 demonstrates the prediction results of the advanced detection method DAFNet based on different depth data. For simple scenes like Image 1, depth maps generated by various monocular depth estimation methods show similar and relatively high quality, enabling DAFNet to accurately identify camouflaged targets. However, for complex scenes like Image 2, some generated depth maps exhibit poor quality or inconsistency with RGB image foregrounds, leading to suboptimal detection results. For instance, DAFNet encounters incomplete target detection issues when using depth maps generated by Depth Pro, while depth maps from Depth-Anything-V2 cause background regions to be misidentified as foreground. Furthermore, for multi-target images like Image 3, most MDE methods struggle to completely estimate depth information for all targets, preventing detection models from achieving comprehensive identification of all camouflaged targets. In contrast, detection methods using Depth _cal can effectively identify camouflaged targets in these challenging scenarios, primarily benefiting from Depth _cal ’s higher depth quality and significantly improved foreground consistency after calibration.

Figure 7:

Typical examples of different depth data (D-A-v2 represents Depth-Anything-V2).

Figure 8:

Visual comparison of detection results obtained by models using different depth data (D-A-v2 represents Depth-Anything-V2).

3.3 Ablation studies

Our DSC framework primarily consists of two core modules: evaluation-based selection and depth calibration. To systematically validate the effectiveness of each module and its components, we conducted two groups of ablation experiments on three benchmark datasets.

1. Effectiveness of evaluation-based selection.

To assess the effectiveness of the evaluation-based selection module, we compared the detection performance when models use Depth _raw (depth data from the initial RGBD COD dataset) versus depth maps generated by various MDE methods. Quantitative results are shown in Tables 1 and 2. Across three datasets, compared to Depth Pro, Depth _raw improved DAFNet’s S _α , F _β , and E _φ metrics by an average of 1.3 %, 1.2 %, and 1.5 % respectively, while reducing M by 0.5 %. Depth _raw , a collection of high-quality depth maps selected through evaluation from those generated by various MDE methods, provides reliable depth information for the COD task. Notably, compared to other depth data, detection models showed smaller performance improvements when using Depth _raw versus depth maps generated by Depth-Anything-V2. This phenomenon is reasonable since depth maps generated by Depth-Anything-V2 inherently possess relatively high quality.

1. Effectiveness of depth calibration.

To evaluate the effectiveness of the depth calibration module, we compared the performance of detection models using Depth _raw , Depth _cal , and Depth’ _cal respectively. Here, Depth’ _cal represents depth data obtained by removing Stage 1 from the TDC strategy, where Depth _raw is directly used as supervision information for the image generation network. It should be noted that removing Stage 2 alone or removing both stages from the TDC strategy would not generate new depth data; in these cases, the depth data after TDC strategy processing remains as Depth _raw . As shown in Table 2, experimental results indicate that detection models experience performance degradation when using Depth’ _cal compared to using Depth _raw . This performance deterioration primarily occurs because Depth _raw contains some low-quality depth maps, and directly using all Depth _raw data as supervision information reduces the quality of generated pseudo depth images, leading to suboptimal detection results. This phenomenon confirms the necessity of Stage 1 in the TDC strategy, which plays a crucial role in generating high-quality calibrated depth maps. Further analysis reveals that all detection models achieve optimal performance when using Depth _cal compared to using Depth _raw and Depth’ _cal , fully validating the effectiveness of the TDC strategy. To intuitively demonstrate the effectiveness of the TDC strategy, we provide visual comparison results of Depth _raw , Depth _cal , and Depth’ _cal in Figure 9. As shown in Figure 9(d), due to using initial depth maps containing low quality and foreground inconsistencies as supervision information, calibrated depth maps generated by the TDC strategy (without Stage 1) still exhibit poor visual quality or inconsistency with corresponding RGB image foregrounds. By comparing Figure 9(c)–(e), we can clearly observe that the complete TDC strategy generates depth images with significantly improved quality. This indicates that our proposed TDC strategy can effectively generate high-quality depth maps with foreground consistency, providing more reliable depth information support for the COD task.

Figure 9:

Visualization results for validating the effectiveness of TDC strategy.

3.4 Generalization experiments

To verify the generalization capability of the proposed framework, we applied it to the RGB-D SOD field. The widely used RGB-D SOD dataset – NJU2K dataset [33] – was selected for experiments. Following the experimental procedure described in this paper, we obtained depth maps generated by various MDE methods as well as the depth calibration results. Relevant examples are shown in Figure 10. By comparing the first six columns in Figure 10, it can be observed that, compared with the ground truth depth maps provided by the dataset (third column), the depth maps generated by MDE methods display better visual effects. Further comparison of the last four columns in Figure 10 shows that the depth calibration results produced by the proposed framework are of higher quality than the original outputs of each MDE method, not only containing more complete 3D scene information and object structural features, but also exhibiting better consistency with the RGB images in terms of foreground alignment. These experimental results indicate that the proposed framework is not only applicable to camouflaged object detection, but can also be effectively used to optimize depth data for other visual tasks, fully demonstrating the method’s strong domain transferability and generalization performance.

Figure 10:

Examples of different depth data on the NJU2K dataset (D-A-v2 represents Depth-Anything-V2).