Analysis of objective quality metrics in computed tomography images affected by metal artifacts
-
Yakdiel Rodriguez-Gallo
and
Marlen Perez-Diaz
Abstract
Image quality (IQ) assessment plays an important role in the medical world. New methods to evaluate image quality have been developed, but their application in the context of computer tomography is yet limited. In this paper the performance of fifteen well-known full reference (FR) IQ metrics is compared with human judgment using images affected by metal artifacts and processed with metal artifact reduction methods from a phantom. Five region of interest with different sizes were selected. IQ was evaluated by seven experienced radiologists completely blinded to the information. To measure the correlation between FR-IQ, and the score assigned by radiologists non-parametric Spearman rank-order correlation coefficient and Kendall’s Rank-order Correlation coefficient were used; so as root mean square error and the mean absolute error to measure the prediction accuracy. Cohen’s kappa was employed with the purpose of assessing inter-observer agreement. The metrics GMSD, IWMSE, IWPSNR, WSNR and OSS-PSNR were the best ranked. Inter-observer agreement was between 0.596 and 0.954, with p<0.001 in all study. The objective scores predicted by these methods correlate consistently with the subjective evaluations. The application of this metrics will make possible a better evaluation of metal artifact reduction algorithms in future works.
Acknowledgments
The authors thank Dr. Luc Beaulieu of Laval University, Québec, Canada, for his help and provide the phantom data set.
-
Research funding: No funding was received for this paper.
-
Author contributions: The authors of this manuscript declare no relationships with any company, whose products or services may be related to the subject matter of the article. All authors have actively contributed to this article through data collection, development of experiments, analysis of results and writing the article.
-
Conflicts of interest: The authors declare that they have no conflict of interest.
-
Informed consent: This article does not contain any studies with human participants or animals performed by any of the authors.
-
Ethical approval: This article does not contain patient data.
References
1. Gjesteby, L, Man, BD, Jin, Y, et al.. Metal artifact reduction in CT: where are we after four decades? IEEE Access 2016;4:5826–49. https://doi.org/10.1109/access.2016.2608621.Search in Google Scholar
2. Jang, J, Bang, K, Jang, H, Hwang, D. Quality evaluation of no-reference MR images using multidirectional filters and image statistics. Magn Reson Med 2018;80:914–24. https://doi.org/10.1002/mrm.27084.Search in Google Scholar PubMed
3. Bognár, G. A No-reference image quality metric with application in low-dose human lung CT image processing. Int J Adv Telecommun Electrotech Signals Syst 2016;5:1–7.10.11601/ijates.v5i1.136Search in Google Scholar
4. Chow, LS, Paramesran, R. Review of medical image quality assessment. Biomed Signal Process Control 2016;27:145–54. https://doi.org/10.1016/j.bspc.2016.02.006.Search in Google Scholar
5. Zhang, Y, Yan, H, Jia, X, Yang, J, Jiang, SB, Mou, X. A hybrid metal artifact reduction algorithm for x-ray CT. Med Phys 2013;40:4. https://doi.org/10.1118/1.4794474.Search in Google Scholar PubMed
6. Veldkamp, WJH, Joemai, RMS, van der Molen, AJ, Geleijns, J. Development and validation of segmentation and interpolation techniques in sinograms for metal artifact suppression in CT. Med Phys 2010;37:620–8. https://doi.org/10.1118/1.3276777.Search in Google Scholar PubMed
7. Hung, S-C, Wu, C-C, Lin, C-J, et al.. Artifact reduction of different metallic implants in flat detector C-arm CT. Am J Neuroradiol 2014;35:1288–92. https://doi.org/10.3174/ajnr.a3851.Search in Google Scholar
8. Meinel, FG, Bischoff, B, Zhang, Q, Bamberg, F, Reiser, MF, Johnson, TRC. Metal artifact reduction by dual-energy computed tomography using energetic extrapolation: a systematically optimized protocol. Invest Radiol 2012;47:406. https://doi.org/10.1097/rli.0b013e31824c86a3.Search in Google Scholar PubMed
9. Prell, D, Kyriakou, Y, Kachelrie, M, Kalender, WA. Reducing metal artifacts in computed tomography caused by hip endoprostheses using a physics-based approach. Invest Radiol 2010;45:747–54. https://doi.org/10.1097/rli.0b013e3181e94384.Search in Google Scholar PubMed
10. Abdoli, M, Mehranian, A, Ailianou, A, Becker, M, Zaidi, H. Assessment of metal artifact reduction methods in pelvic CT. Med Phys 2016;43:1588–97. https://doi.org/10.1118/1.4942810.Search in Google Scholar PubMed
11. Yazdi, M, Lari, MA, Bernier, G, Beaulieu, L. An opposite view data replacement approach for reducing artifacts due to metallic dental objects. Med Phys 2011;38:2275–81. https://doi.org/10.1118/1.3566016.Search in Google Scholar PubMed
12. Meyer, E, Raupach, R, Lell, M, Schmidt, B, Kachelriess, M, Kachelrieß, M. Frequency split metal artifact reduction (FSMAR) in computed tomography. Med Phys 2012;39:1904. https://doi.org/10.1118/1.3691902.Search in Google Scholar PubMed
13. Zhang, X, Xing, L. Sequentially reweighted TV minimization for CT metal artifact reduction. Med Phys 2013;40:71907. https://doi.org/10.1118/1.4811129.Search in Google Scholar PubMed PubMed Central
14. Park, HS, Hwang, D, Seo, JK. Metal artifact reduction for polychromatic X-ray CT based on a beam-hardening corrector. IEEE Trans Med Imag 2016;35:480–7. https://doi.org/10.1109/tmi.2015.2478905.Search in Google Scholar
15. Zhang, H, Wang, L, Li, L, Cai, A, Hu, G, Yan, B. Iterative metal artifact reduction for x-ray computed tomography using unmatched projector/backprojector pairs. Med Phys 2016;43:3019–33. https://doi.org/10.1118/1.4950722.Search in Google Scholar PubMed
16. Bazalova, M, Beaulieu, L, Palefsky, S, Verhaegen, F. Correction of CT artifacts and its influence on Monte Carlo dose calculations. Med Phys 2007;34:2119–32. https://doi.org/10.1118/1.2736777.Search in Google Scholar PubMed
17. Huang, VW, Kohli, K. Evaluation of new commercially available metal artifact reduction (MAR) algorithm on both image quality and relative dosimetry for patients with hip prosthesis or dental fillings. Int J Med Phys Clin Eng Radiat Oncol 2017;6:124. https://doi.org/10.4236/ijmpcero.2017.62012.Search in Google Scholar
18. Guilfoile, C, Rampant, P, House, M. The impact of smart metal artefact reduction algorithm for use in radiotherapy treatment planning. Australas Phys Eng Sci Med 2017;40:385–94. https://doi.org/10.1007/s13246-017-0543-5.Search in Google Scholar PubMed
19. Xu, C, Verhaegen, F, Laurendeau, D, Enger, SA, Beaulieu, L. An algorithm for efficient metal artifact reductions in permanent seed implants. Med Phys 2011;38:47. https://doi.org/10.1118/1.3519988.Search in Google Scholar PubMed
20. Morsbach, F, Bickelhaupt, S, Wanner, GA, Krauss, A, Schmidt, B, Alkadhi, H. Reduction of metal artifacts from hip prostheses on CT images of the pelvis: value of iterative reconstructions. Radiology 2013;268:237–44. https://doi.org/10.1148/radiol.13122089.Search in Google Scholar PubMed
21. Wang, Z, Bovik, AC. A universal image quality index. IEEE Signal Process Lett 2002;9:81–4. https://doi.org/10.1109/97.995823.Search in Google Scholar
22. Wang, Z, Bovik, AC, Sheikh, HR, Simoncelli, EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004;13:600–12. https://doi.org/10.1109/tip.2003.819861.Search in Google Scholar PubMed
23. Zhang, L, Zhang, L, Mou, X, Zhang, D. FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 2011;20:2378–86. https://doi.org/10.1109/tip.2011.2109730.Search in Google Scholar PubMed
24. Larson, EC, Chandler, DM. Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electron Imag 2010;19:011006.10.1117/12.810071Search in Google Scholar
25. Xue, W, Zhang, L, Mou, X, Bovik, AC. Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans Image Process 2014;23:684–95. https://doi.org/10.1109/tip.2013.2293423.Search in Google Scholar PubMed
26. Wang, Z, Li, Q. Information content weighting for perceptual image quality assessment. IEEE Trans Image Process 2011;20:1185–98. https://doi.org/10.1109/tip.2010.2092435.Search in Google Scholar
27. Damera-Venkata, N, Kite, TD, Geisler, WS, Evans, BL, Bovik, AC. Image quality assessment based on a degradation model. IEEE Trans Image Process 2000;9:636–50. https://doi.org/10.1109/83.841940.Search in Google Scholar PubMed
28. Mannos, J, Sakrison, D. The effects of a visual fidelity criterion of the encoding of images. IEEE Trans Inf Theory 1974;20:525–36. https://doi.org/10.1109/tit.1974.1055250.Search in Google Scholar
29. Gu, K, Liu, M, Zhai, G, Yang, X, Zhang, W. Quality assessment considering viewing distance and image resolution. IEEE Trans Broadcast 2015;61:520–31. https://doi.org/10.1109/tbc.2015.2459851.Search in Google Scholar
30. Zhang, L, Li, H. SR-SIM: a fast and high performance IQA index based on spectral residual. In: 19th IEEE International Conference on Image Processing; 2012:1473–6 pp.10.1109/ICIP.2012.6467149Search in Google Scholar
31. Yang, G, Li, D, Lu, F, Liao, Y, Yang, W. RVSIM: a feature similarity method for full-reference image quality assessment. J Image Video Proc 2018;2018:6. https://doi.org/10.1186/s13640-018-0246-1.Search in Google Scholar
32. Yazdia, M, Gingras, L, Beaulieu, L. An adaptive approach to metal artifact reduction in helical computed tomography for radiation therapy treatment planning: experimental and clinical studies. Int J Radiat Oncol Biol Phys 2005;62:1224–31. https://doi.org/10.1016/j.ijrobp.2005.02.052.Search in Google Scholar PubMed
33. Kalender, WA, Hebel, R, Ebersberger, J. Reduction of CT artifacts caused by metallic implants. Radiol 1987;164:576–7. https://doi.org/10.1148/radiology.164.2.3602406.Search in Google Scholar PubMed
34. Rodríguez-Gallo, Y, Orozco-Morales, R, Pérez-Díaz, M. Metal artifact reduction by morphological image filtering for computed tomography. In: World Congress on Medical Physics and Biomedical Engineering 2018, Singapore. Springer; 2019:219–22 pp.10.1007/978-981-10-9035-6_39Search in Google Scholar
35. Meyer, E, Raupach, R, Lell, M, Schmidt, B, Kachelrieß, M. Normalized metal artifact reduction (NMAR) in computed tomography. Med Phys 2010;37:5482–93. https://doi.org/10.1118/1.3484090.Search in Google Scholar PubMed
36. Rodríguez-Gallo, Y, Orozco-Morales, R, Pérez-Díaz, M. Gradient image smoothing for metal artifact reduction (GISMAR) in computed tomography. Biomed Phys Eng Express 2019;5:035012.10.1088/2057-1976/ab0c4dSearch in Google Scholar
37. Fleiss, JL, Cohen, J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 1973;33:613–9. https://doi.org/10.1177/001316447303300309.Search in Google Scholar
38. Landis, JR, Koch, GG. The measurement of observer agreement for categorical data. Biometrics; 1977;33:159–74. https://doi.org/10.2307/2529310.Search in Google Scholar
39. Barton, B, Peat, J. Medical statistics: a guide to SPSS, data analysis and critical appraisal. New York: John Wiley & Sons; 2014.Search in Google Scholar
40. Sheikh, HR, Sabir, MF, Bovik, AC. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Proc 2006;15:3440–51. https://doi.org/10.1109/tip.2006.881959.Search in Google Scholar PubMed
41. Pedersen, M, Hardeberg, JY. Full-reference image quality metrics: classification and evaluation. Found Trends Comput Graph Vis 2012;7:1–80.10.1561/9781601985279Search in Google Scholar
42. Willmott, CJ, Matsuura, K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res 2005;30:79–82. https://doi.org/10.3354/cr030079.Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Research Articles
- Analysis of objective quality metrics in computed tomography images affected by metal artifacts
- Endoscopic confocal laser-microscopy for the intraoperative nerve recognition: is it feasible?
- A fuzzy sensitivity analysis approach to estimate brain effective connectivity and its application to epileptic seizure detection
- Comparative study of platinum electroplating to improve micro gold electrode arrays with LCP laminate
- Stresses and deformations of an osteosynthesis plate in a lateral tibia plateau fracture
- Biomechanical influence of thread form on stress distribution over short implants (≤6 mm) using finite element analysis
Articles in the same Issue
- Frontmatter
- Research Articles
- Analysis of objective quality metrics in computed tomography images affected by metal artifacts
- Endoscopic confocal laser-microscopy for the intraoperative nerve recognition: is it feasible?
- A fuzzy sensitivity analysis approach to estimate brain effective connectivity and its application to epileptic seizure detection
- Comparative study of platinum electroplating to improve micro gold electrode arrays with LCP laminate
- Stresses and deformations of an osteosynthesis plate in a lateral tibia plateau fracture
- Biomechanical influence of thread form on stress distribution over short implants (≤6 mm) using finite element analysis
