Evaluation of the International Monitoring System and International Data Centre of the Comprehensive Nuclear-Test-Ban Treaty Organization

Appendix 1. Evaluation of IDC peak search algorithm

The quality of the peak search algorithm is of vital importance for the detection capability of IDC data processing. Small peaks should be found, while at the same time, the number of spurious peaks (type I errors) should be minimized. A thorough evaluation of the peak search algorithm was carried out in late 2000 at the PTS by E. Arik and H. Toivonen [9].

The most commonly used peak search algorithms are based on Mariscotti method [11]. This was implemented in the SAMPO mainframe computer program already in late sixties. Since then it is widely used by commercial software, such as Sampo 80, Sampo 90, UniSampo, Canberra Genie-PC and Gamma-Vision. The Canberra implementation used at the PTS is very near that of Sampo 80. All these applications try to find peaks where the second derivative reaches a minimum. Statistical uncertainty is reduced by smoothing, i. e., several channels are considered simultaneously. The key parameter, the Peak Search Sensitivity Threshold (ST), controls the sensitivity of the search algorithm. A peak is considered as found if the sensitivity parameter exceeds the preset threshold.

The PTS study had three goals:

• Develop a tool to evaluate the performance of the peak search algorithm utilizing artificial spectra with known information contents.

• Validate the tool by comparing results with those of real spectra.

• Perform parameter sensitivity analysis to optimize the detection capability against false alarms.

The evaluation was based on the quantity of peak significance S which is defined as

where A is the peak area and BA is the baseline area.

The width of the ROI is considered to cover + 3σ, where σ is the standard deviation of the Gaussian function describing the peak. Adopting the concept of full-width half-maximum (FWHM = 2.355σ), the width of the ROI, expressed in channels, is close to 2.5*FWHM at the energy involved. Therefore, for constant baseline of BC counts per channel, the baseline area is

Generation of artificial spectra

Artificial spectra were generated using Monte Carlo techniques. The spectra have 8192 channels with desired baseline counts BC per channel. B_C is fixed to a constant of 100 or 10 over the entire range of the spectrum. For each spectrum, 100 artificial photo-peaks were generated with Gaussian shape. The peaks were superimposed on top of the baseline at regular intervals of 27 keV.

Once the ideal smooth spectrum was prepared, each channel content was varied randomly according to Poisson distribution with mean value being the number of theoretical counts of that channel. The area A, and consequently, the peak significance S were other input variables of the artificial spectra. For each S value between 0.2 and 2.0 (in steps of 0.2), 100 spectra were generated.

Each peak is labeled as real or spurious using a lookup table of input peak centroids and their areas. Therefore, for every ST setting, the average number of real peaks found in each one of the 100 spectra is easy to calculate.

Estimation of peak detection probability

The probability of finding real peaks per spectrum (p) can be plotted as a function of S (Fig. 1).

A group of 100 artificial spectra, containing no real peaks and through having a special B_C = IDC 100 test counts pipeline per channel, with ST 3.0were , 2.5analyzed , 2.0, 1.5 and 1.0. The results are shown in Table 1.

The result of the peak search algorithm should be considered as a candidate list of peaks, not as a final list. In automated spectrum processing, a low value could be given for the peak search sensitivity threshold (e. g ST = 2.5). Some of the potential peaks would then be rejected automatically using a preset cut-off for peak significance (e. g. S = 0.6).

Validation of the evaluation method

Natural radionuclides ²¹²Pb, ²¹²Bi and ²⁰⁸Tl, are always identified in IMS spectra which are measured obeying IMS requirements (24 h sampling, 24 h decay and 24 h acquisition).

The large peaks are well quantified by IDC software. This information and appropriate data correction procedures, coincidence correction in particular, allow calculating peak areas of other peaks. Keeping track which of these smaller peaks are found and which are missed provides information to calculate the detection probability as a function of peak significance.

The calculations were performed with the software package Eval-Shaman. The tool is utilizing Shaman [7] which reads peak data from the IDC database and produces tailored reports for further processing. The results were practically identical with those (Fig. 1) obtained using artificial spectra with constant baseline of 100 counts per channel [9].