ECALL: Estimation-calibrated learning for joint deconvolution and kernel inference from unpaired data
-
Markus Haltmeier
and
Gyeongha Hwang
Abstract
Recovering an original image from a blurred observation when the blurring kernel is unknown is a classical inverse problem with applications in astronomy, microscopy, and medical imaging. This setting is often referred to as blind deconvolution, a highly ill-posed problem that suffers from significant non-uniqueness and typically requires strong prior assumptions or training data. While recent deep learning methods have achieved impressive performance, they generally rely on access to paired datasets of original and blurred images, which is often difficult to acquire. In this paper, we propose ECALL (Estimation-Calibrated Learning), a novel framework for joint kernel and signal recovery from unpaired data. ECALL operates without paired observations, instead using separate collections of original and blurred images. The method leverages moment-based constraints to estimate the kernel and simultaneously perform deconvolution. A key component of our approach is a loss function that incorporates cycle consistency with respect to the estimated kernel and a reconstruction operator, alongside terms that match statistical properties (e.g., moments) of the data distributions. We demonstrate the effectiveness of ECALL with numerical experiments, highlighting its potential in scenarios where paired data are unavailable.
Funding statement: Gyeongha Hwang was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (NRF-2021R1F1A1048120).
References
[1] E. Agustsson and R. Timofte, Ntire 2017 challenge on single image super-resolution: Dataset and study, Proceedings of the IEEE conference on Computer Vision and Pattern Recognition Workshops, IEEE Press, Piscataway (2017), 126–135. 10.1109/CVPRW.2017.150Search in Google Scholar
[2] S. D. Babacan, R. Molina, M. N. Do and A. K. Katsaggelos, Bayesian blind deconvolution with general sparse image priors, Computer Vision–ECCV 2012, Springer, Berlin (2012), 341–355. 10.1007/978-3-642-33783-3_25Search in Google Scholar
[3] D. O. Baguer, J. Leuschner and M. Schmidt, Computed tomography reconstruction using deep image prior and learned reconstruction methods, Inverse Problems 36 (2020), no. 9, Article ID 094004. 10.1088/1361-6420/aba415Search in Google Scholar
[4] G. Boracchi and A. Foi, Modeling the performance of image restoration from motion blur, IEEE Trans. Image Process. 21 (2012), no. 8, 3502–3517. 10.1109/TIP.2012.2192126Search in Google Scholar PubMed
[5] A. Chakrabarti, A neural approach to blind motion deblurring, Computer Vision–ECCV 2016, Springer, Berlin (2016), 221–235. 10.1007/978-3-319-46487-9_14Search in Google Scholar
[6] T. F. Chan and C.-K. Wong, Total variation blind deconvolution, IEEE Trans. Image Process. 7 (1998), no. 3, 370–375. 10.1109/83.661187Search in Google Scholar PubMed
[7] S. Cho and S. Lee, Fast motion deblurring, ACM Trans. Graphics 28 (2009), 1–8. 10.1145/1618452.1618491Search in Google Scholar
[8] Y. Choi, Y. Uh, J. Yoo and J.-W. Ha, Stargan v2: Diverse image synthesis for multiple domains, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2020), 8188–8197. 10.1109/CVPR42600.2020.00821Search in Google Scholar
[9] A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta and A. A. Bharath, Generative adversarial networks: An overview, IEEE Signal Process. Mag. 35 (2018), 53–65. 10.1109/MSP.2017.2765202Search in Google Scholar
[10] R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis and W. T. Freeman, Removing camera shake from a single photograph, ACM Trans. Graphics 25 (2006), no. 3, 787–794. 10.1145/1141911.1141956Search in Google Scholar
[11] Y. Gandelsman, A. Shocher and M. Irani, Double-DIP: Unsupervised image decomposition via coupled deep-image-priors, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2019), 11026–11035. 10.1109/CVPR.2019.01128Search in Google Scholar
[12] J. P. Gardner, J. C. Mather, M. Clampin, R. Doyon, M. A. Greenhouse, H. B. Hammel, J. B. Hutchings, P. Jakobsen, S. J. Lilly and K. S. Long, The James Webb space telescope, Space Sci. Rev. 123 (2006), no. 4, 485–606. 10.1007/s11214-006-8315-7Search in Google Scholar
[13] A. V. Gončarskiĭ, A. S. Leonov and A. G. Jagola, Certain estimates of the rate of convergence of regularized approximations for equations of convolution type, USSR Comput. Math. Math. Phys. 12 (1972), no. 3, 243–254. 10.1016/0041-5553(72)90046-8Search in Google Scholar
[14] D. Gong, M. Tan, Y. Zhang, A. Van den Hengel and Q. Shi, Blind image deconvolution by automatic gradient activation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2016), 1827–1836. 10.1109/CVPR.2016.202Search in Google Scholar
[15] D. Gong, J. Yang, L. Liu, Y. Zhang, I. Reid, C. Shen, A. Van Den Hengel and Q. Shi, From motion blur to motion flow: A deep learning solution for removing heterogeneous motion blur, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2017), 2319–2328. 10.1109/CVPR.2017.405Search in Google Scholar
[16] L. He, A. Marquina and S. J. Osher, Blind deconvolution using tv regularization and bregman iteration, Int. J. Imag. Syst. Technol. 15 (2005), 74–83. 10.1002/ima.20040Search in Google Scholar
[17] V. K. Ivanov, V. V. Vasin and V. P. Tanana, Theory of Linear Ill-Posed Problems and its Applications, De Gruyter, Moscow, 2013. Search in Google Scholar
[18] L. Justen and R. Ramlau, A non-iterative regularization approach to blind deconvolution, Inverse Problems 22 (2006), no. 3, 771–800. 10.1088/0266-5611/22/3/003Search in Google Scholar
[19] D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, preprint (2014), https://arxiv.org/abs/1412.6980. Search in Google Scholar
[20] D. Krishnan and R. Fergus, Fast image deconvolution using hyper-Laplacian priors, Proceedings of the 23rd International Conference on Neural Information Processing Systems, ACM, New York (2009), 1033–1041. Search in Google Scholar
[21] D. Krishnan, T. Tay and R. Fergus, Blind deconvolution using a normalized sparsity measure, CVPR 2011, IEEE Press, Piscataway (2011), 233–240. 10.1109/CVPR.2011.5995521Search in Google Scholar
[22] D. Kundur and D. Hatzinakos, Blind image deconvolution, IEEE Signal Process. Mag. 13 (1996), no. 3, 43–64. 10.1109/79.489268Search in Google Scholar
[23] O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin and J. Matas, Deblurgan: Blind motion deblurring using conditional adversarial networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2018), 8183–8192. 10.1109/CVPR.2018.00854Search in Google Scholar
[24] O. Kupyn, T. Martyniuk, J. Wu and Z. Wang, Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better, Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE Press, Piscataway (2019), 8878–8887. 10.1109/ICCV.2019.00897Search in Google Scholar
[25] A. Levin, Y. Weiss, F. Durand and W. T. Freeman, Understanding and evaluating blind deconvolution algorithms, 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2009), 1964–1971. 10.1109/CVPRW.2009.5206815Search in Google Scholar
[26] A. Levin, Y. Weiss, F. Durand and W. T. Freeman, Efficient marginal likelihood optimization in blind deconvolution, CVPR 2011, IEEE Press, Piscataway (2011), 2657–2664. 10.1109/CVPR.2011.5995308Search in Google Scholar
[27] J. Li, Y. Nan and H. Ji, Un-supervised learning for blind image deconvolution via Monte-Carlo sampling, Inverse Problems 38 (2022), no. 3, Article ID 035012. 10.1088/1361-6420/ac4edeSearch in Google Scholar
[28] R. Liu and J. Jia, Reducing boundary artifacts in image deconvolution, 2008 15th IEEE International Conference on Image Processing, IEEE Press, Piscataway (2008), 505–508. 10.1109/ICIP.2008.4711802Search in Google Scholar
[29] I. Loshchilov and F. Hutter, Decoupled weight decay regularization, preprint (2017), https://arxiv.org/abs/1711.05101. Search in Google Scholar
[30] B. Lu, J.-C. Chen and R. Chellappa, Unsupervised domain-specific deblurring via disentangled representations, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2019), 10225–10234. 10.1109/CVPR.2019.01047Search in Google Scholar
[31] X.-G. Lv, F. Li and T. Zeng, Convex blind image deconvolution with inverse filtering, Inverse Problems 34 (2018), no. 3, Article ID 035003. 10.1088/1361-6420/aaa4a7Search in Google Scholar
[32] A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution, SIAM J. Imaging Sci. 2 (2009), no. 1, 64–83. 10.1137/080724289Search in Google Scholar
[33] M.-E. Nilsback and A. Zisserman, Automated flower classification over a large number of classes, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, IEEE Press, Piscataway (2008), 722–729. 10.1109/ICVGIP.2008.47Search in Google Scholar
[34] M. Noroozi, P. Chandramouli and P. Favaro, Motion deblurring in the wild, Pattern Recognition, Lecture Notes in Comput. Sci. 10496, Springer, Cham (2017), 65–77. 10.1007/978-3-319-66709-6_6Search in Google Scholar
[35] J. Pan, D. Sun, H. Pfister and M.-H. Yang, Blind image deblurring using dark channel prior, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2016), 1628–1636. 10.1109/CVPR.2016.180Search in Google Scholar
[36] S. Ramakrishnan, S. Pachori, A. Gangopadhyay and S. Raman, Deep generative filter for motion deblurring, Proceedings of the IEEE International Conference on Computer Vision Workshops, IEEE Press, Piscataway (2017), 2993–3000. 10.1109/ICCVW.2017.353Search in Google Scholar
[37] D. Ren, K. Zhang, Q. Wang, Q. Hu and W. Zuo, Neural blind deconvolution using deep priors, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2020), 3341–3350. 10.1109/CVPR42600.2020.00340Search in Google Scholar
[38] O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015, Springer, Cham (2015), 234–241. 10.1007/978-3-319-24574-4_28Search in Google Scholar
[39] L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D 60 (1992), no. 1–4, 259–268. 10.1016/0167-2789(92)90242-FSearch in Google Scholar
[40] B. D. Savage and K. R. Sembach, Interstellar abundances from absorption-line observations with the hubble space telescope, Ann. Rev. Astronom. Astrophys. 34 (1996), 279–329. 10.1146/annurev.astro.34.1.279Search in Google Scholar
[41] O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier and F. Lenzen, Variational Methods in Imaging, Appl. Math. Sci. 167, Springer, New York, 2009. Search in Google Scholar
[42] C. J. Schuler, M. Hirsch, S. Harmeling and B. Schölkopf, Learning to deblur, IEEE Trans. Pattern Anal. Mach. Intell. 38 (2015), no. 7, 1439–1451. 10.1109/TPAMI.2015.2481418Search in Google Scholar PubMed
[43] Q. Shan, J. Jia and A. Agarwala, High-quality motion deblurring from a single image, ACM Trans. Graphics 27 (2008), no. 3, 1–10. 10.1145/1360612.1360672Search in Google Scholar
[44] J. Sun, W. Cao, Z. Xu and J. Ponce, Learning a convolutional neural network for non-uniform motion blur removal, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2015), 769–777. 10.1109/CVPR.2015.7298677Search in Google Scholar
[45] X. Tao, H. Gao, X. Shen, J. Wang and J. Jia, Scale-recurrent network for deep image deblurring, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2018), 8174–8182. 10.1109/CVPR.2018.00853Search in Google Scholar
[46] D. Ulyanov, A. Vedaldi and V. Lempitsky, Deep image prior, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2018), 9446–9454. 10.1109/CVPR.2018.00984Search in Google Scholar
[47] V. N. Vapnik, The Nature of Statistical Learning Theory, Springer, New York, 1999. 10.1007/978-1-4757-3264-1Search in Google Scholar
[48] D. Wipf and H. Zhang, Revisiting Bayesian blind deconvolution, J. Mach. Learn. Res. 15 (2014), 3595–3634. Search in Google Scholar
[49] L. Xu and J. Jia, Two-phase kernel estimation for robust motion deblurring, Computer Vision–ECCV 2010, Springer, Berlin (2010), 157–170. 10.1007/978-3-642-15549-9_12Search in Google Scholar
[50] L. Xu, J. S. Ren, C. Liu and J. Jia, Deep convolutional neural network for image deconvolution, Proceedings of the 28th International Conference on Neural Information Processing Systems – Volume 1, ACM, New York (2014), 1790–1798. Search in Google Scholar
[51] L. Xu, S. Zheng and J. Jia, Unnatural l0 sparse representation for natural image deblurring, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2013), 1107–1114. 10.1109/CVPR.2013.147Search in Google Scholar
[52] C. Yu, B. Li, J. Xiao, C. Sun, S. Tang, C. Bi, C. Cui and D. Fan, Astronomical data fusion: Recent progress and future prospects – a survey, Exp. Astronomy 47 (2019), no. 3, 359–380. 10.1007/s10686-019-09633-zSearch in Google Scholar
[53] J. Zhang, J. Pan, J. Ren, Y. Song, L. Bao, R. W. Lau and M.-H. Yang, Dynamic scene deblurring using spatially variant recurrent neural networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, Piscataway (2018), 2521–2529. 10.1109/CVPR.2018.00267Search in Google Scholar
[54] Y. Zhang and C. Chen, Stochastic linear regularization methods: random discrepancy principleand applications, Inverse Problems 40 (2024), no. 2, Article ID 025007. 10.1088/1361-6420/ad149eSearch in Google Scholar
[55] Y. Zhang, K. Li, K. Li, B. Zhong and Y. Fu, Residual non-local attention networks for image restoration, preprint (2019), https://arxiv.org/abs/1903.10082. Search in Google Scholar
[56] Y. Zhang, D. V. Lukyanenko and A. G. Yagola, An optimal regularization method for convolution equations on the sourcewise represented set, J. Inverse Ill-Posed Probl. 23 (2015), no. 5, 465–475. 10.1515/jiip-2014-0047Search in Google Scholar
[57] J.-Y. Zhu, T. Park, P. Isola and A. A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, Proceedings of the IEEE International Conference on Computer Vision, IEEE Press, Piscataway (2017), 2223–2232. 10.1109/ICCV.2017.244Search in Google Scholar
© 2025 Walter de Gruyter GmbH, Berlin/Boston
